You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by pw...@apache.org on 2014/07/10 20:11:06 UTC
git commit: HOTFIX: Minor doc update for sbt change
Repository: spark
Updated Branches:
refs/heads/master 628932b8d -> 88006a623
HOTFIX: Minor doc update for sbt change
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/88006a62
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/88006a62
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/88006a62
Branch: refs/heads/master
Commit: 88006a62377d2b7c9886ba49ceef158737bc1b97
Parents: 628932b
Author: Patrick Wendell <pw...@gmail.com>
Authored: Thu Jul 10 11:10:43 2014 -0700
Committer: Patrick Wendell <pw...@gmail.com>
Committed: Thu Jul 10 11:11:00 2014 -0700
----------------------------------------------------------------------
README.md | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/88006a62/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 6211a58..01ef851 100644
--- a/README.md
+++ b/README.md
@@ -69,29 +69,28 @@ can be run using:
Spark uses the Hadoop core library to talk to HDFS and other Hadoop-supported
storage systems. Because the protocols have changed in different versions of
Hadoop, you must build Spark against the same version that your cluster runs.
-You can change the version by setting the `SPARK_HADOOP_VERSION` environment
-when building Spark.
+You can change the version by setting `-Dhadoop.version` when building Spark.
For Apache Hadoop versions 1.x, Cloudera CDH MRv1, and other Hadoop
versions without YARN, use:
# Apache Hadoop 1.2.1
- $ SPARK_HADOOP_VERSION=1.2.1 sbt/sbt assembly
+ $ sbt/sbt -Dhadoop.version=1.2.1 assembly
# Cloudera CDH 4.2.0 with MapReduce v1
- $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly
+ $ sbt/sbt -Dhadoop.version=2.0.0-mr1-cdh4.2.0 assembly
For Apache Hadoop 2.2.X, 2.1.X, 2.0.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions
with YARN, also set `SPARK_YARN=true`:
# Apache Hadoop 2.0.5-alpha
- $ SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt/sbt assembly
+ $ sbt/sbt -Dhadoop.version=2.0.5-alpha -Pyarn assembly
# Cloudera CDH 4.2.0 with MapReduce v2
- $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly
+ $ sbt/sbt -Dhadoop.version=2.0.0-cdh4.2.0 -Pyarn assembly
# Apache Hadoop 2.2.X and newer
- $ SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt assembly
+ $ sbt/sbt -Dhadoop.version=2.2.0 -Pyarn assembly
When developing a Spark application, specify the Hadoop version by adding the
"hadoop-client" artifact to your project's dependencies. For example, if you're