You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2017/08/28 22:19:25 UTC
Review Request 61956: HIVE-17323
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/
-----------------------------------------------------------
Review request for hive, Gopal V and Jason Dere.
Bugs: HIVE-17323
https://issues.apache.org/jira/browse/HIVE-17323
Repository: hive-git
Description
-------
HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
https://issues.apache.org/jira/browse/HIVE-16260
However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
Diff: https://reviews.apache.org/r/61956/diff/1/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 61956: HIVE-17323
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On Aug. 29, 2017, 6:28 p.m., Gopal V wrote:
> > ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
> > Line 2667 (original), 2893 (patched)
> > <https://reviews.apache.org/r/61956/diff/2/?file=1807067#file1807067line2893>
> >
> > Result change?
>
> Deepak Jaiswal wrote:
> There are two changes to the tests.
> 1. An accidental copy-paste mistake led to same test being used twice, that was fixed causing the results to change. (notice the query modification in .q file)
> 2. Added a new test for which new results are added to the output.
Oh wait, I missed this, let me take a look.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/#review184080
-----------------------------------------------------------
On Aug. 29, 2017, 6:24 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61956/
> -----------------------------------------------------------
>
> (Updated Aug. 29, 2017, 6:24 p.m.)
>
>
> Review request for hive, Gopal V and Jason Dere.
>
>
> Bugs: HIVE-17323
> https://issues.apache.org/jira/browse/HIVE-17323
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
> https://issues.apache.org/jira/browse/HIVE-16260
> However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
> ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
>
>
> Diff: https://reviews.apache.org/r/61956/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 61956: HIVE-17323
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On Aug. 29, 2017, 6:28 p.m., Gopal V wrote:
> > ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
> > Line 2667 (original), 2893 (patched)
> > <https://reviews.apache.org/r/61956/diff/2/?file=1807067#file1807067line2893>
> >
> > Result change?
There are two changes to the tests.
1. An accidental copy-paste mistake led to same test being used twice, that was fixed causing the results to change. (notice the query modification in .q file)
2. Added a new test for which new results are added to the output.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/#review184080
-----------------------------------------------------------
On Aug. 29, 2017, 6:24 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61956/
> -----------------------------------------------------------
>
> (Updated Aug. 29, 2017, 6:24 p.m.)
>
>
> Review request for hive, Gopal V and Jason Dere.
>
>
> Bugs: HIVE-17323
> https://issues.apache.org/jira/browse/HIVE-17323
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
> https://issues.apache.org/jira/browse/HIVE-16260
> However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
> ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
>
>
> Diff: https://reviews.apache.org/r/61956/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 61956: HIVE-17323
Posted by Gopal V <go...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/#review184080
-----------------------------------------------------------
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
Line 2667 (original), 2893 (patched)
<https://reviews.apache.org/r/61956/#comment260076>
Result change?
- Gopal V
On Aug. 29, 2017, 6:24 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61956/
> -----------------------------------------------------------
>
> (Updated Aug. 29, 2017, 6:24 p.m.)
>
>
> Review request for hive, Gopal V and Jason Dere.
>
>
> Bugs: HIVE-17323
> https://issues.apache.org/jira/browse/HIVE-17323
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
> https://issues.apache.org/jira/browse/HIVE-16260
> However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
> ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
>
>
> Diff: https://reviews.apache.org/r/61956/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 61956: HIVE-17323
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/
-----------------------------------------------------------
(Updated Aug. 30, 2017, 2:21 a.m.)
Review request for hive, Gopal V and Jason Dere.
Changes
-------
Redid the code to track the parallel semijoin edges when it encounters a MapJoinOperator in small table side.
Bugs: HIVE-17323
https://issues.apache.org/jira/browse/HIVE-17323
Repository: hive-git
Description
-------
HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
https://issues.apache.org/jira/browse/HIVE-16260
However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
Diff: https://reviews.apache.org/r/61956/diff/4/
Changes: https://reviews.apache.org/r/61956/diff/3-4/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 61956: HIVE-17323
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/
-----------------------------------------------------------
(Updated Aug. 29, 2017, 7:28 p.m.)
Review request for hive, Gopal V and Jason Dere.
Changes
-------
Updated the test to work with and without semijoin reduction for better comparison.
The result is different because it is working on a smaller table created with only 40 rows.
Bugs: HIVE-17323
https://issues.apache.org/jira/browse/HIVE-17323
Repository: hive-git
Description
-------
HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
https://issues.apache.org/jira/browse/HIVE-16260
However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
Diff: https://reviews.apache.org/r/61956/diff/3/
Changes: https://reviews.apache.org/r/61956/diff/2-3/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 61956: HIVE-17323
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61956/
-----------------------------------------------------------
(Updated Aug. 29, 2017, 6:24 p.m.)
Review request for hive, Gopal V and Jason Dere.
Changes
-------
Added a fix for ConcurrentModificationException and using correct way to implement DFS.
Bugs: HIVE-17323
https://issues.apache.org/jira/browse/HIVE-17323
Repository: hive-git
Description
-------
HIVE-16260 allows removal of parallel edges of semijoin with mapjoins.
https://issues.apache.org/jira/browse/HIVE-16260
However, it should also consider dynamic partition pruning edge like semijoin without removing it while traversing the query tree.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 1671773d4a
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q b22890bc9d
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out 478b0828a3
Diff: https://reviews.apache.org/r/61956/diff/2/
Changes: https://reviews.apache.org/r/61956/diff/1-2/
Testing
-------
Thanks,
Deepak Jaiswal