You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jimmy Xiang <jx...@cloudera.com> on 2014/12/09 20:41:59 UTC
Review Request 28864: HIVE-9042 Support multiple mapjoin operators in
one work [Spark Branch]
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/
-----------------------------------------------------------
Review request for hive and Xuefu Zhang.
Bugs: HIVE-9042
https://issues.apache.org/jira/browse/HIVE-9042
Repository: hive-git
Description
-------
In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
Diff: https://reviews.apache.org/r/28864/diff/
Testing
-------
Unit test.
Thanks,
Jimmy Xiang
Re: Review Request 28864: HIVE-9042 Support multiple mapjoin
operators in one work [Spark Branch]
Posted by Jimmy Xiang <jx...@cloudera.com>.
> On Dec. 9, 2014, 8:28 p.m., Xuefu Zhang wrote:
> > Patch looks a good. A test case with multiple map joins in one work would be awesome.
Let me add a test case. Thanks.
> On Dec. 9, 2014, 8:28 p.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 94
> > <https://reviews.apache.org/r/28864/diff/1/?file=787121#file787121line94>
> >
> > There is no need to be "lazy" here. We can just instantiate it here.
Just trying to avoid creating an object in case not needed.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/#review64428
-----------------------------------------------------------
On Dec. 9, 2014, 7:41 p.m., Jimmy Xiang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28864/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 7:41 p.m.)
>
>
> Review request for hive and Xuefu Zhang.
>
>
> Bugs: HIVE-9042
> https://issues.apache.org/jira/browse/HIVE-9042
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
> ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
>
> Diff: https://reviews.apache.org/r/28864/diff/
>
>
> Testing
> -------
>
> Unit test.
>
>
> Thanks,
>
> Jimmy Xiang
>
>
Re: Review Request 28864: HIVE-9042 Support multiple mapjoin
operators in one work [Spark Branch]
Posted by Jimmy Xiang <jx...@cloudera.com>.
> On Dec. 9, 2014, 8:28 p.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 94
> > <https://reviews.apache.org/r/28864/diff/1/?file=787121#file787121line94>
> >
> > There is no need to be "lazy" here. We can just instantiate it here.
>
> Jimmy Xiang wrote:
> Just trying to avoid creating an object in case not needed.
Fixed it.
- Jimmy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/#review64428
-----------------------------------------------------------
On Dec. 9, 2014, 7:41 p.m., Jimmy Xiang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28864/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 7:41 p.m.)
>
>
> Review request for hive and Xuefu Zhang.
>
>
> Bugs: HIVE-9042
> https://issues.apache.org/jira/browse/HIVE-9042
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
> ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
>
> Diff: https://reviews.apache.org/r/28864/diff/
>
>
> Testing
> -------
>
> Unit test.
>
>
> Thanks,
>
> Jimmy Xiang
>
>
Re: Review Request 28864: HIVE-9042 Support multiple mapjoin
operators in one work [Spark Branch]
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/#review64428
-----------------------------------------------------------
Patch looks a good. A test case with multiple map joins in one work would be awesome.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java
<https://reviews.apache.org/r/28864/#comment107135>
There is no need to be "lazy" here. We can just instantiate it here.
- Xuefu Zhang
On Dec. 9, 2014, 7:41 p.m., Jimmy Xiang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28864/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 7:41 p.m.)
>
>
> Review request for hive and Xuefu Zhang.
>
>
> Bugs: HIVE-9042
> https://issues.apache.org/jira/browse/HIVE-9042
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
> ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
>
> Diff: https://reviews.apache.org/r/28864/diff/
>
>
> Testing
> -------
>
> Unit test.
>
>
> Thanks,
>
> Jimmy Xiang
>
>
Re: Review Request 28864: HIVE-9042 Support multiple mapjoin
operators in one work [Spark Branch]
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/#review64464
-----------------------------------------------------------
Ship it!
- Xuefu Zhang
On Dec. 9, 2014, 11:20 p.m., Jimmy Xiang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28864/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2014, 11:20 p.m.)
>
>
> Review request for hive and Xuefu Zhang.
>
>
> Bugs: HIVE-9042
> https://issues.apache.org/jira/browse/HIVE-9042
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
>
>
> Diffs
> -----
>
> itests/src/test/resources/testconfiguration.properties d6f8267
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
> ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
> ql/src/test/queries/clientpositive/bucket_map_join_spark4.q PRE-CREATION
> ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/28864/diff/
>
>
> Testing
> -------
>
> Unit test.
>
>
> Thanks,
>
> Jimmy Xiang
>
>
Re: Review Request 28864: HIVE-9042 Support multiple mapjoin
operators in one work [Spark Branch]
Posted by Jimmy Xiang <jx...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28864/
-----------------------------------------------------------
(Updated Dec. 9, 2014, 11:20 p.m.)
Review request for hive and Xuefu Zhang.
Changes
-------
Addressed Xuefu's review comments. Thanks.
Bugs: HIVE-9042
https://issues.apache.org/jira/browse/HIVE-9042
Repository: hive-git
Description
-------
In one spark work, there could be multiple mapjoin operators, but only one could be bucket mapjoin.
Diffs (updated)
-----
itests/src/test/resources/testconfiguration.properties d6f8267
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java 6977abc
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java ef04885
ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java 2976c3a
ql/src/test/queries/clientpositive/bucket_map_join_spark4.q PRE-CREATION
ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out PRE-CREATION
ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/28864/diff/
Testing
-------
Unit test.
Thanks,
Jimmy Xiang