You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by pw...@apache.org on 2014/06/08 10:25:05 UTC
git commit: SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop
Version
Repository: spark
Updated Branches:
refs/heads/master 7b877b270 -> a338834f9
SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
The Maven Profiles that refer to hadoopX, e.g. `hadoop2.4`, should set the expected
`hadoop.version` and `yarn.version`.
e.g.
```
<profile>
<id>hadoop-2.4</id>
<properties>
<hadoop.version>2.4.0</hadoop.version>
<yarn.version>${hadoop.version}</yarn.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
</properties>
</profile>
```
Builds can still define the `-Dhadoop.version` option but this will correctly default the
Hadoop Version to the one that is expected according the profile that is selected.
e.g.
```$ mvn -P hadoop-2.4,yarn clean install```
or
```$ mvn -P hadoop-0.23,yarn clean install```
[ticket] : https://issues.apache.org/jira/browse/SPARK-2026
Author : berngp
Reviewer : ?
Author: Bernardo Gomez Palacio <be...@gmail.com>
Closes #998 from berngp/feature/SPARK-2026 and squashes the following commits:
07ba4f7 [Bernardo Gomez Palacio] SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a338834f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a338834f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a338834f
Branch: refs/heads/master
Commit: a338834f90556d78119b37985b603ecee85f97ed
Parents: 7b877b2
Author: Bernardo Gomez Palacio <be...@gmail.com>
Authored: Sun Jun 8 01:24:52 2014 -0700
Committer: Patrick Wendell <pw...@gmail.com>
Committed: Sun Jun 8 01:24:52 2014 -0700
----------------------------------------------------------------------
pom.xml | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a338834f/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 891468b..0d46bb4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -209,14 +209,14 @@
<repository>
<id>spring-releases</id>
<name>Spring Release Repository</name>
- <url>http://repo.spring.io/libs-release</url>
+ <url>http://repo.spring.io/libs-release</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
- </repository>
+ </repository>
</repositories>
<dependencyManagement>
@@ -987,11 +987,15 @@
<artifactId>avro</artifactId>
</dependency>
</dependencies>
+ <properties>
+ <hadoop.version>0.23.10</hadoop.version>
+ </properties>
</profile>
<profile>
<id>hadoop-2.2</id>
<properties>
+ <hadoop.version>2.2.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
</properties>
</profile>
@@ -999,6 +1003,7 @@
<profile>
<id>hadoop-2.3</id>
<properties>
+ <hadoop.version>2.3.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
</properties>
@@ -1007,6 +1012,7 @@
<profile>
<id>hadoop-2.4</id>
<properties>
+ <hadoop.version>2.4.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
</properties>