You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2017/01/31 03:01:08 UTC
Review Request 56110: Fix ConcurrentModification Excpetion in
HIVE-15760
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56110/
-----------------------------------------------------------
Review request for hive and Jason Dere.
Bugs: HIVE-15760
https://issues.apache.org/jira/browse/HIVE-15760
Repository: hive-git
Description
-------
Fix ConcurrentModification Excpetion when removing semijoin operator due to a cycle created by Mapjoin-Mapjoin-semijoin.
It now traverses the graph and prepares a list of potential candidates and then processes them in 2nd pass.
Did the similar changes for SMBJoin related semijoin removal logic.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java cf8e843
Diff: https://reviews.apache.org/r/56110/diff/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 56110: Fix ConcurrentModification Excpetion in
HIVE-15760
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On Jan. 31, 2017, 8:43 a.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java, line 672
> > <https://reviews.apache.org/r/56110/diff/1/?file=1619700#file1619700line672>
> >
> > Another option here might have been to put more of the checking into the NodeProcessor implementation, and return just the items that need to be removed.
Both ways have their pros and cons.
In the current method, the NodeProcessor does nothing but make a list, however, this makes the code more readable as the actual logic to remove semijoin happens in the same function.
I did implement the code as suggested first but later on moved to current state due to the above reason.
If you insist, I will update the code.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56110/#review163636
-----------------------------------------------------------
On Jan. 31, 2017, 3:01 a.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56110/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 3:01 a.m.)
>
>
> Review request for hive and Jason Dere.
>
>
> Bugs: HIVE-15760
> https://issues.apache.org/jira/browse/HIVE-15760
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Fix ConcurrentModification Excpetion when removing semijoin operator due to a cycle created by Mapjoin-Mapjoin-semijoin.
>
> It now traverses the graph and prepares a list of potential candidates and then processes them in 2nd pass.
> Did the similar changes for SMBJoin related semijoin removal logic.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java cf8e843
>
> Diff: https://reviews.apache.org/r/56110/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 56110: Fix ConcurrentModification Excpetion in
HIVE-15760
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56110/#review163636
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java (line 663)
<https://reviews.apache.org/r/56110/#comment235113>
Another option here might have been to put more of the checking into the NodeProcessor implementation, and return just the items that need to be removed.
- Jason Dere
On Jan. 31, 2017, 3:01 a.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56110/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 3:01 a.m.)
>
>
> Review request for hive and Jason Dere.
>
>
> Bugs: HIVE-15760
> https://issues.apache.org/jira/browse/HIVE-15760
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Fix ConcurrentModification Excpetion when removing semijoin operator due to a cycle created by Mapjoin-Mapjoin-semijoin.
>
> It now traverses the graph and prepares a list of potential candidates and then processes them in 2nd pass.
> Did the similar changes for SMBJoin related semijoin removal logic.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java cf8e843
>
> Diff: https://reviews.apache.org/r/56110/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>