You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2020/03/12 16:09:43 UTC
[spark] branch branch-2.4 updated: [SPARK-31130][BUILD] Use the
same version of `commons-io` in SBT
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new e6bcaaa [SPARK-31130][BUILD] Use the same version of `commons-io` in SBT
e6bcaaa is described below
commit e6bcaaa8a78f6c88b5c5276e90cd049e2cffc658
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Thu Mar 12 09:06:29 2020 -0700
[SPARK-31130][BUILD] Use the same version of `commons-io` in SBT
This PR (SPARK-31130) aims to pin `Commons IO` version to `2.4` in SBT build like Maven build.
[HADOOP-15261](https://issues.apache.org/jira/browse/HADOOP-15261) upgraded `commons-io` from 2.4 to 2.5 at Apache Hadoop 3.1.
In `Maven`, Apache Spark always uses `Commons IO 2.4` based on `pom.xml`.
```
$ git grep commons-io.version
pom.xml: <commons-io.version>2.4</commons-io.version>
pom.xml: <version>${commons-io.version}</version>
```
However, `SBT` choose `2.5`.
**branch-3.0**
```
$ build/sbt -Phadoop-3.2 "core/dependencyTree" | grep commons-io:commons-io | head -n1
[info] | | +-commons-io:commons-io:2.5
```
**branch-2.4**
```
$ build/sbt -Phadoop-3.1 "core/dependencyTree" | grep commons-io:commons-io | head -n1
[info] | | +-commons-io:commons-io:2.5
```
No.
Pass the Jenkins with `[test-hadoop3.2]` (the default PR Builder is `SBT`) and manually do the following locally.
```
build/sbt -Phadoop-3.2 "core/dependencyTree" | grep commons-io:commons-io | head -n1
```
Closes #27886 from dongjoon-hyun/SPARK-31130.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit 972e23d18186c73026ebed95b37a886ca6eecf3e)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
project/SparkBuild.scala | 1 +
1 file changed, 1 insertion(+)
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 3f85ac6..7ee079c 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -552,6 +552,7 @@ object DockerIntegrationTests {
object DependencyOverrides {
lazy val settings = Seq(
dependencyOverrides += "com.google.guava" % "guava" % "14.0.1",
+ dependencyOverrides += "commons-io" % "commons-io" % "2.4",
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.6.7.3",
dependencyOverrides += "jline" % "jline" % "2.14.6")
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org