You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sahil Takiar <ta...@gmail.com> on 2016/09/20 19:44:50 UTC
Re: Review Request 48839: HIVE-14029: Update Spark version to 2.0.0
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48839/#review149717
-----------------------------------------------------------
pom.xml (line 179)
<https://reviews.apache.org/r/48839/#comment217403>
Can this be changed to `2.0.0` instead of `2.0.0-preview`
- Sahil Takiar
On June 17, 2016, 8:52 a.m., cheng xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48839/
> -----------------------------------------------------------
>
> (Updated June 17, 2016, 8:52 a.m.)
>
>
> Review request for hive, Rui Li, Sergio Pena, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-14029
> https://issues.apache.org/jira/browse/HIVE-14029
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up Spark to 2.0.0 to benefit those performance improvements.
>
>
> Diffs
> -----
>
> pom.xml 63a5ae1
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveBaseFunctionResultList.java 5b65036
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveMapFunction.java 53c5c0e
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveReduceFunction.java f6595f1
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SortByShuffler.java a6350d3
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/JobMetricsListener.java 09c54c1
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 4b34ebf
> ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestHiveKVResultCache.java ee9f9b7
> spark-client/src/main/java/org/apache/hive/spark/client/MetricsCollection.java e77aa78
> spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java e3b88d1
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/InputMetrics.java e46b67d
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/Metrics.java a7305cf
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleReadMetrics.java be14c06
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java 4420e4d
> spark-client/src/test/java/org/apache/hive/spark/client/TestMetricsCollection.java 5146e91
>
> Diff: https://reviews.apache.org/r/48839/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> cheng xu
>
>
Re: Review Request 48839: HIVE-14029: Update Spark version to 2.0.0
Posted by cheng xu <ch...@intel.com>.
> On Sept. 21, 2016, 3:44 a.m., Sahil Takiar wrote:
> > pom.xml, line 179
> > <https://reviews.apache.org/r/48839/diff/1/?file=1422055#file1422055line179>
> >
> > Can this be changed to `2.0.0` instead of `2.0.0-preview`
>
> Sahil Takiar wrote:
> Looked at your updated patch, seems like you already did this.
I forgot to update the review board entry. Reattach file to update it.
- cheng
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48839/#review149717
-----------------------------------------------------------
On Sept. 21, 2016, 1:27 p.m., cheng xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48839/
> -----------------------------------------------------------
>
> (Updated Sept. 21, 2016, 1:27 p.m.)
>
>
> Review request for hive, Rui Li, Sergio Pena, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-14029
> https://issues.apache.org/jira/browse/HIVE-14029
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up Spark to 2.0.0 to benefit those performance improvements.
>
>
> Diffs
> -----
>
> itests/pom.xml a452db3
> pom.xml 2fb78cd
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveBaseFunctionResultList.java 5b65036
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveMapFunction.java 53c5c0e
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveReduceFunction.java f6595f1
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SortByShuffler.java a6350d3
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/JobMetricsListener.java 09c54c1
> ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestHiveKVResultCache.java ee9f9b7
> spark-client/pom.xml 6cf3b17
> spark-client/src/main/java/org/apache/hive/spark/client/MetricsCollection.java e77aa78
> spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java e3b88d1
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/InputMetrics.java e46b67d
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/Metrics.java a7305cf
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleReadMetrics.java be14c06
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java 4420e4d
> spark-client/src/test/java/org/apache/hive/spark/client/TestMetricsCollection.java 5146e91
>
> Diff: https://reviews.apache.org/r/48839/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> cheng xu
>
>
Re: Review Request 48839: HIVE-14029: Update Spark version to 2.0.0
Posted by Sahil Takiar <ta...@gmail.com>.
> On Sept. 20, 2016, 7:44 p.m., Sahil Takiar wrote:
> > pom.xml, line 179
> > <https://reviews.apache.org/r/48839/diff/1/?file=1422055#file1422055line179>
> >
> > Can this be changed to `2.0.0` instead of `2.0.0-preview`
Looked at your updated patch, seems like you already did this.
- Sahil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48839/#review149717
-----------------------------------------------------------
On June 17, 2016, 8:52 a.m., cheng xu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48839/
> -----------------------------------------------------------
>
> (Updated June 17, 2016, 8:52 a.m.)
>
>
> Review request for hive, Rui Li, Sergio Pena, Szehon Ho, and Xuefu Zhang.
>
>
> Bugs: HIVE-14029
> https://issues.apache.org/jira/browse/HIVE-14029
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> There are quite some new optimizations in Spark 2.0.0. We need to bump up Spark to 2.0.0 to benefit those performance improvements.
>
>
> Diffs
> -----
>
> pom.xml 63a5ae1
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveBaseFunctionResultList.java 5b65036
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveMapFunction.java 53c5c0e
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveReduceFunction.java f6595f1
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SortByShuffler.java a6350d3
> ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/JobMetricsListener.java 09c54c1
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 4b34ebf
> ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestHiveKVResultCache.java ee9f9b7
> spark-client/src/main/java/org/apache/hive/spark/client/MetricsCollection.java e77aa78
> spark-client/src/main/java/org/apache/hive/spark/client/RemoteDriver.java e3b88d1
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/InputMetrics.java e46b67d
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/Metrics.java a7305cf
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleReadMetrics.java be14c06
> spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java 4420e4d
> spark-client/src/test/java/org/apache/hive/spark/client/TestMetricsCollection.java 5146e91
>
> Diff: https://reviews.apache.org/r/48839/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> cheng xu
>
>