You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Remus Rusanu <re...@microsoft.com> on 2013/12/11 01:08:53 UTC
Review Request 16167: HIVE-5595 Implement Vectorized SMB Join
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16167/
-----------------------------------------------------------
Review request for hive, Ashutosh Chauhan, Eric Hanson, and Jitendra Pandey.
Bugs: HIVE-5595
https://issues.apache.org/jira/browse/HIVE-5595
Repository: hive-git
Description
-------
See HIVE-5595 I will post description
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 24a812d
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 81a1232
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 19f7d79
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java 4bfeb20
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java abdc165
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 7859e56
ql/src/test/queries/clientpositive/vectorized_bucketmapjoin1.q PRE-CREATION
ql/src/test/results/clientpositive/vectorized_bucketmapjoin1.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/16167/diff/
Testing
-------
New .q file, manually tested several cases
Thanks,
Remus Rusanu
Re: Review Request 16167: HIVE-5595 Implement Vectorized SMB Join
Posted by Eric Hanson <eh...@microsoft.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16167/#review30289
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57976>
Please remove trailing white space in all your code. You can set the eclipse editor to do this.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57980>
Can you add a comment about the purpose of this class and the major differences from regular SMB Join?
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57985>
Excellent variable names
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57984>
Please add a few comments in the body explaining the major sections
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57983>
Does the fact that this is a map from Byte mean there is a limit of 127 ANDed filter expressions? I guess that is enough for most purposes but it seems like a like internal limit. Not sure if this is a limitation inherited from someplace else.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57987>
need blank after =
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57989>
Sun Java coding standards say put blanks around =, <
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57988>
and replacing them
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57993>
please put blanks around :
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment57998>
good comment!
spell out atm
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment58020>
I don't understand this because the body of the loop does not change for each trip through the loop. It looks like you are doing the same thing inBatch.size times. Is this right? If so, please explain.
Should tag be batchIndex?
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment58021>
Please add comment before method
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java
<https://reviews.apache.org/r/16167/#comment58022>
add blanks around operators
ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java
<https://reviews.apache.org/r/16167/#comment58023>
Please comment the tests to explain what you are checking
- Eric Hanson
On Dec. 11, 2013, 7:26 a.m., Remus Rusanu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16167/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2013, 7:26 a.m.)
>
>
> Review request for hive, Ashutosh Chauhan, Eric Hanson, and Jitendra Pandey.
>
>
> Bugs: HIVE-5595
> https://issues.apache.org/jira/browse/HIVE-5595
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> See HIVE-5595 I will post description
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 24a812d
> ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 81a1232
> ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 19f7d79
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java 4bfeb20
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java abdc165
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 7859e56
> ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java 02031ea
> ql/src/test/queries/clientpositive/vectorized_bucketmapjoin1.q PRE-CREATION
> ql/src/test/results/clientpositive/vectorized_bucketmapjoin1.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/16167/diff/
>
>
> Testing
> -------
>
> New .q file, manually tested several cases
>
>
> Thanks,
>
> Remus Rusanu
>
>
Re: Review Request 16167: HIVE-5595 Implement Vectorized SMB Join
Posted by Remus Rusanu <re...@microsoft.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16167/
-----------------------------------------------------------
(Updated Jan. 13, 2014, 2:14 p.m.)
Review request for hive, Ashutosh Chauhan, Eric Hanson, and Jitendra Pandey.
Bugs: HIVE-5595
https://issues.apache.org/jira/browse/HIVE-5595
Repository: hive-git
Description
-------
See HIVE-5595 I will post description
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 24a812d
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 81a1232
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java fccea89
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java d189dde
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 32fd191
ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java 02031ea
ql/src/test/queries/clientpositive/vectorized_bucketmapjoin1.q PRE-CREATION
ql/src/test/results/clientpositive/vectorized_bucketmapjoin1.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/16167/diff/
Testing
-------
New .q file, manually tested several cases
Thanks,
Remus Rusanu
Re: Review Request 16167: HIVE-5595 Implement Vectorized SMB Join
Posted by Remus Rusanu <re...@microsoft.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16167/
-----------------------------------------------------------
(Updated Dec. 11, 2013, 7:26 a.m.)
Review request for hive, Ashutosh Chauhan, Eric Hanson, and Jitendra Pandey.
Bugs: HIVE-5595
https://issues.apache.org/jira/browse/HIVE-5595
Repository: hive-git
Description
-------
See HIVE-5595 I will post description
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 24a812d
ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java 81a1232
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 19f7d79
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSMBMapJoinOperator.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java 4bfeb20
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java abdc165
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java 7859e56
ql/src/test/org/apache/hadoop/hive/ql/optimizer/physical/TestVectorizer.java 02031ea
ql/src/test/queries/clientpositive/vectorized_bucketmapjoin1.q PRE-CREATION
ql/src/test/results/clientpositive/vectorized_bucketmapjoin1.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/16167/diff/
Testing
-------
New .q file, manually tested several cases
Thanks,
Remus Rusanu