You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2016/11/02 18:52:32 UTC
spark git commit: [SPARK-17058][BUILD] Add maven
snapshots-and-staging profile to build/test against staging artifacts
Repository: spark
Updated Branches:
refs/heads/master 3c24299b7 -> 37d95227a
[SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts
## What changes were proposed in this pull request?
Adds a `snapshots-and-staging profile` so that RCs of projects like Hadoop and HBase can be used in developer-only build and test runs. There's a comment above the profile telling people not to use this in production.
There's no attempt to do the same for SBT, as Ivy is different.
## How was this patch tested?
Tested by building against the Hadoop 2.7.3 RC 1 JARs
without the profile (and without any local copy of the 2.7.3 artifacts), the build failed
```
mvn install -DskipTests -Pyarn,hadoop-2.7,hive -Dhadoop.version=2.7.3
...
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project Launcher 2.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.pom
[WARNING] The POM for org.apache.hadoop:hadoop-client:jar:2.7.3 is missing, no dependency information available
Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 4.482 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 17.402 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 11.252 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 13.458 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 9.043 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 16.027 s]
[INFO] Spark Project Launcher ............................. FAILURE [ 1.653 s]
[INFO] Spark Project Core ................................. SKIPPED
...
```
With the profile, the build completed
```
mvn install -DskipTests -Pyarn,hadoop-2.7,hive,snapshots-and-staging -Dhadoop.version=2.7.3
```
Author: Steve Loughran <st...@apache.org>
Closes #14646 from steveloughran/stevel/SPARK-17058-support-asf-snapshots.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/37d95227
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/37d95227
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/37d95227
Branch: refs/heads/master
Commit: 37d95227a21de602b939dae84943ba007f434513
Parents: 3c24299
Author: Steve Loughran <st...@apache.org>
Authored: Wed Nov 2 11:52:29 2016 -0700
Committer: Reynold Xin <rx...@databricks.com>
Committed: Wed Nov 2 11:52:29 2016 -0700
----------------------------------------------------------------------
pom.xml | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/37d95227/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index aaf7cfa..04d2eaa 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2694,6 +2694,54 @@
</profile>
<!--
+ This is a profile to enable the use of the ASF snapshot and staging repositories
+ during a build. It is useful when testing againt nightly or RC releases of dependencies.
+ It MUST NOT be used when building copies of Spark to use in production of for distribution,
+ -->
+ <profile>
+ <id>snapshots-and-staging</id>
+ <properties>
+ <!-- override point for ASF staging/snapshot repos -->
+ <asf.staging>https://repository.apache.org/content/groups/staging/</asf.staging>
+ <asf.snapshots>https://repository.apache.org/content/repositories/snapshots/</asf.snapshots>
+ </properties>
+
+ <pluginRepositories>
+ <pluginRepository>
+ <id>ASF Staging</id>
+ <url>${asf.staging}</url>
+ </pluginRepository>
+ <pluginRepository>
+ <id>ASF Snapshots</id>
+ <url>${asf.snapshots}</url>
+ <snapshots>
+ <enabled>true</enabled>
+ </snapshots>
+ <releases>
+ <enabled>false</enabled>
+ </releases>
+ </pluginRepository>
+
+ </pluginRepositories>
+ <repositories>
+ <repository>
+ <id>ASF Staging</id>
+ <url>${asf.staging}</url>
+ </repository>
+ <repository>
+ <id>ASF Snapshots</id>
+ <url>${asf.snapshots}</url>
+ <snapshots>
+ <enabled>true</enabled>
+ </snapshots>
+ <releases>
+ <enabled>false</enabled>
+ </releases>
+ </repository>
+ </repositories>
+ </profile>
+
+ <!--
These empty profiles are available in some sub-modules. Declare them here so that
maven does not complain when they're provided on the command line for a sub-module
that does not have them.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org