You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/05/03 22:22:37 UTC
spark git commit: [SPARK-7302] [DOCS] SPARK building documentation
still mentions building for yarn 0.23
Repository: spark
Updated Branches:
refs/heads/master daa70bf13 -> 9e25b09f8
[SPARK-7302] [DOCS] SPARK building documentation still mentions building for yarn 0.23
Remove references to Hadoop 0.23
CC tgravescs Is this what you had in mind? basically all refs to 0.23?
We don't support YARN 0.23, but also don't support Hadoop 0.23 anymore AFAICT. There are no builds or releases for it.
In fact, on a related note, refs to CDH3 (Hadoop 0.20.2) should be removed as this certainly isn't supported either.
Author: Sean Owen <so...@cloudera.com>
Closes #5863 from srowen/SPARK-7302 and squashes the following commits:
42f5d1e [Sean Owen] Remove CDH3 (Hadoop 0.20.2) refs too
dad02e3 [Sean Owen] Remove references to Hadoop 0.23
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9e25b09f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9e25b09f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9e25b09f
Branch: refs/heads/master
Commit: 9e25b09f8809378777ae8bbe75dca12d2c45ff4c
Parents: daa70bf
Author: Sean Owen <so...@cloudera.com>
Authored: Sun May 3 21:22:31 2015 +0100
Committer: Sean Owen <so...@cloudera.com>
Committed: Sun May 3 21:22:31 2015 +0100
----------------------------------------------------------------------
docs/building-spark.md | 4 ----
docs/hadoop-third-party-distributions.md | 3 ---
make-distribution.sh | 2 +-
pom.xml | 14 --------------
4 files changed, 1 insertion(+), 22 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/9e25b09f/docs/building-spark.md
----------------------------------------------------------------------
diff --git a/docs/building-spark.md b/docs/building-spark.md
index ea79c5b..287fcd3 100644
--- a/docs/building-spark.md
+++ b/docs/building-spark.md
@@ -66,7 +66,6 @@ Because HDFS is not protocol-compatible across versions, if you want to read fro
<tr><th>Hadoop version</th><th>Profile required</th></tr>
</thead>
<tbody>
- <tr><td>0.23.x</td><td>hadoop-0.23</td></tr>
<tr><td>1.x to 2.1.x</td><td>(none)</td></tr>
<tr><td>2.2.x</td><td>hadoop-2.2</td></tr>
<tr><td>2.3.x</td><td>hadoop-2.3</td></tr>
@@ -82,9 +81,6 @@ mvn -Dhadoop.version=1.2.1 -DskipTests clean package
# Cloudera CDH 4.2.0 with MapReduce v1
mvn -Dhadoop.version=2.0.0-mr1-cdh4.2.0 -DskipTests clean package
-
-# Apache Hadoop 0.23.x
-mvn -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean package
{% endhighlight %}
You can enable the "yarn" profile and optionally set the "yarn.version" property if it is different from "hadoop.version". Spark only supports YARN versions 2.2.0 and later.
http://git-wip-us.apache.org/repos/asf/spark/blob/9e25b09f/docs/hadoop-third-party-distributions.md
----------------------------------------------------------------------
diff --git a/docs/hadoop-third-party-distributions.md b/docs/hadoop-third-party-distributions.md
index 87dcc58..96bd69c 100644
--- a/docs/hadoop-third-party-distributions.md
+++ b/docs/hadoop-third-party-distributions.md
@@ -29,9 +29,6 @@ the _exact_ Hadoop version you are running to avoid any compatibility errors.
<tr><th>Release</th><th>Version code</th></tr>
<tr><td>CDH 4.X.X (YARN mode)</td><td>2.0.0-cdh4.X.X</td></tr>
<tr><td>CDH 4.X.X</td><td>2.0.0-mr1-cdh4.X.X</td></tr>
- <tr><td>CDH 3u6</td><td>0.20.2-cdh3u6</td></tr>
- <tr><td>CDH 3u5</td><td>0.20.2-cdh3u5</td></tr>
- <tr><td>CDH 3u4</td><td>0.20.2-cdh3u4</td></tr>
</table>
</td>
<td>
http://git-wip-us.apache.org/repos/asf/spark/blob/9e25b09f/make-distribution.sh
----------------------------------------------------------------------
diff --git a/make-distribution.sh b/make-distribution.sh
index 92177e1..1bfa9ac 100755
--- a/make-distribution.sh
+++ b/make-distribution.sh
@@ -58,7 +58,7 @@ while (( "$#" )); do
--hadoop)
echo "Error: '--hadoop' is no longer supported:"
echo "Error: use Maven profiles and options -Dhadoop.version and -Dyarn.version instead."
- echo "Error: Related profiles include hadoop-0.23, hdaoop-2.2, hadoop-2.3 and hadoop-2.4."
+ echo "Error: Related profiles include hadoop-2.2, hadoop-2.3 and hadoop-2.4."
exit_with_usage
;;
--with-yarn)
http://git-wip-us.apache.org/repos/asf/spark/blob/9e25b09f/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 4313f94..de18741 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1615,20 +1615,6 @@
-->
<profile>
- <id>hadoop-0.23</id>
- <!-- SPARK-1121: Adds an explicit dependency on Avro to work around a Hadoop 0.23.X issue -->
- <dependencies>
- <dependency>
- <groupId>org.apache.avro</groupId>
- <artifactId>avro</artifactId>
- </dependency>
- </dependencies>
- <properties>
- <hadoop.version>0.23.10</hadoop.version>
- </properties>
- </profile>
-
- <profile>
<id>hadoop-2.2</id>
<properties>
<hadoop.version>2.2.0</hadoop.version>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org