You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/13 04:42:46 UTC
[GitHub] [spark] sfcoy opened a new pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
sfcoy opened a new pull request #30022:
URL: https://github.com/apache/spark/pull/30022
* Compatible with Hadoop > 3.2.0
* Future proof for a while
### What changes were proposed in this pull request?
Upgrade the Google Guava dependency for compatibility with Hadoop 3.2.1 and Hadoop 3.3.0.
### Why are the changes needed?
Spark fails at runtime with NoSuchMethodExceptions when built/run in conjunction with these versions, which make use of com.google.common.base.Preconditions methods that are not present in the version of Guava currently specified for Spark.
### Does this PR introduce _any_ user-facing change?
This change introduces new dependencies into the build which are imported by the guava pom file.
### How was this patch tested?
We are currently running ETL production processes using Spark builds with this Guava version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-758818171
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38564/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708699576
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129752/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707483670
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708106738
> After [SPARK-29250](https://issues.apache.org/jira/browse/SPARK-29250), I guess this PR will be a general Guava version upgrade PR without any relation to `Hadoop 3.2.1`.
IMO, if spark-3 with hadoop3.2 can work well in Hadoop cluster (2.6/2/7/2.8. etc), it's ok just use hadoop 3.2 client.
In our hadoop cluster, we use spark-2.4-hadoop-2.6 run in hadoop-3.2.1 cluster, works well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708673440
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34358/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-758779898
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707849045
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708698814
**[Test build #129752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129752/testReport)** for PR 30022 at commit [`3ff577e`](https://github.com/apache/spark/commit/3ff577eedb70a594ff2d54fd99f0495b7e07a6df).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sfcoy commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
sfcoy commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-759289382
The Kubernetes integration test appears to be running out of disk space:
`Step 5/18 : COPY jars /opt/spark/jars
failed to copy files: failed to copy directory: Error processing tar file(exit status 1): write /kubernetes-model-networking-4.10.3.jar: no space left on device`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-758810239
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38564/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #30022:
URL: https://github.com/apache/spark/pull/30022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708601752
**[Test build #129752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129752/testReport)** for PR 30022 at commit [`3ff577e`](https://github.com/apache/spark/commit/3ff577eedb70a594ff2d54fd99f0495b7e07a6df).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708699569
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707771282
The big big problem here is that previous Hadoop versions (<= 3.2.0) use Guava 14 or so, so this would break some compatibility with them. I think it could only happen for a Hadoop 3.2.1+ profile, but, there may be a good idea.
We'd still have to figure out whether it breaks compatibility with other libs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #30022:
URL: https://github.com/apache/spark/pull/30022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707853981
BTW, @sfcoy and @AngersZhuuuu .
For the following, Apache Spark community wants to use the official Hadoop 3 client to cut the dependency dramatically.
> Upgrade the Google Guava dependency for compatibility with Hadoop 3.2.1 and Hadoop 3.3.0.
Please see here.
- https://github.com/apache/spark/pull/29843 (SPARK-29250 Upgrade to Hadoop 3.2.1 and move to shaded client)
After SPARK-29250, I guess this PR will be a general Guava version upgrade PR without any relation to `Hadoop 3.2.1`.
cc @sunchao
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707520794
FYI @dongjoon-hyun @srowen
How about we make guava version change with hadoop version in profile?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707850088
Thanks, @sfcoy . This is an interesting approach.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708601752
**[Test build #129752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129752/testReport)** for PR 30022 at commit [`3ff577e`](https://github.com/apache/spark/commit/3ff577eedb70a594ff2d54fd99f0495b7e07a6df).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707484086
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708673459
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-758812809
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38564/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707483670
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708699569
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-826002233
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708665664
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34358/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707484086
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-758779898
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-708673459
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sfcoy commented on pull request #30022: SPARK-33090 Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
sfcoy commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-707532612
> FYI @dongjoon-hyun @srowen
> How about we make guava version change with hadoop version in profile?
Hi @AngersZhuuuu , I'm not sure I see any benefit in that. It will increase the complexity of an already complicated build system. It's significantly more than just a version number change. If you look at the changed files you will see what I mean.
Complexity is the enemy of maintainability.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #30022: [SPARK-33090][BUILD][test-hadoop2.7] Upgrade Google Guava to 29.0-jre
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #30022:
URL: https://github.com/apache/spark/pull/30022#issuecomment-709482519
Not sure if this works well with Hive 2.3.x also since it is still on Guava 14.0.1.
> IMO, if spark-3 with hadoop3.2 can work well in Hadoop cluster (2.6/2/7/2.8. etc), it's ok just use hadoop 3.2 client.
Yes it's expected to work. There is an issue [HDFS-15191](https://issues.apache.org/jira/browse/HDFS-15191) which potentially breaks compatibility between Hadoop 3.2.1 and 2.x server but it is fixed in 3.2.2 (which Spark is probably going to use).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org