You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2016/03/19 05:32:52 UTC
spark git commit: [MINOR][DOCS] Update build descriptions and commands
Repository: spark
Updated Branches:
refs/heads/master f43a26ef9 -> c11ea2e41
[MINOR][DOCS] Update build descriptions and commands
## What changes were proposed in this pull request?
This PR updates Scala and Hadoop versions in the build description and commands in `Building Spark` documents.
## How was this patch tested?
N/A
Author: Dongjoon Hyun <do...@apache.org>
Closes #11838 from dongjoon-hyun/fix_doc_building_spark.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c11ea2e4
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c11ea2e4
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c11ea2e4
Branch: refs/heads/master
Commit: c11ea2e4138acdd8d4ed487049ded35346bca528
Parents: f43a26e
Author: Dongjoon Hyun <do...@apache.org>
Authored: Fri Mar 18 21:32:48 2016 -0700
Committer: Reynold Xin <rx...@databricks.com>
Committed: Fri Mar 18 21:32:48 2016 -0700
----------------------------------------------------------------------
docs/building-spark.md | 11 +++++++----
docs/index.md | 4 ++--
docs/running-on-mesos.md | 4 ++--
docs/running-on-yarn.md | 4 ++--
4 files changed, 13 insertions(+), 10 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/c11ea2e4/docs/building-spark.md
----------------------------------------------------------------------
diff --git a/docs/building-spark.md b/docs/building-spark.md
index e478954..1e202ac 100644
--- a/docs/building-spark.md
+++ b/docs/building-spark.md
@@ -98,8 +98,11 @@ mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package
# Apache Hadoop 2.4.X or 2.5.X
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=VERSION -DskipTests clean package
-Versions of Hadoop after 2.5.X may or may not work with the -Phadoop-2.4 profile (they were
-released after this version of Spark).
+# Apache Hadoop 2.6.X
+mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
+
+# Apache Hadoop 2.7.X and later
+mvn -Pyarn -Phadoop-2.7 -Dhadoop.version=VERSION -DskipTests clean package
# Different versions of HDFS and YARN.
mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Dyarn.version=2.2.0 -DskipTests clean package
@@ -140,10 +143,10 @@ It's possible to build Spark sub-modules using the `mvn -pl` option.
For instance, you can build the Spark Streaming module using:
{% highlight bash %}
-mvn -pl :spark-streaming_2.10 clean install
+mvn -pl :spark-streaming_2.11 clean install
{% endhighlight %}
-where `spark-streaming_2.10` is the `artifactId` as defined in `streaming/pom.xml` file.
+where `spark-streaming_2.11` is the `artifactId` as defined in `streaming/pom.xml` file.
# Continuous Compilation
http://git-wip-us.apache.org/repos/asf/spark/blob/c11ea2e4/docs/index.md
----------------------------------------------------------------------
diff --git a/docs/index.md b/docs/index.md
index 9dfc52a..20eab56 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -130,8 +130,8 @@ options for deployment:
* [StackOverflow tag `apache-spark`](http://stackoverflow.com/questions/tagged/apache-spark)
* [Mailing Lists](http://spark.apache.org/mailing-lists.html): ask questions about Spark here
* [AMP Camps](http://ampcamp.berkeley.edu/): a series of training camps at UC Berkeley that featured talks and
- exercises about Spark, Spark Streaming, Mesos, and more. [Videos](http://ampcamp.berkeley.edu/3/),
- [slides](http://ampcamp.berkeley.edu/3/) and [exercises](http://ampcamp.berkeley.edu/3/exercises/) are
+ exercises about Spark, Spark Streaming, Mesos, and more. [Videos](http://ampcamp.berkeley.edu/6/),
+ [slides](http://ampcamp.berkeley.edu/6/) and [exercises](http://ampcamp.berkeley.edu/6/exercises/) are
available online for free.
* [Code Examples](http://spark.apache.org/examples.html): more are also available in the `examples` subfolder of Spark ([Scala]({{site.SPARK_GITHUB_URL}}/tree/master/examples/src/main/scala/org/apache/spark/examples),
[Java]({{site.SPARK_GITHUB_URL}}/tree/master/examples/src/main/java/org/apache/spark/examples),
http://git-wip-us.apache.org/repos/asf/spark/blob/c11ea2e4/docs/running-on-mesos.md
----------------------------------------------------------------------
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md
index 3a832de..293a828 100644
--- a/docs/running-on-mesos.md
+++ b/docs/running-on-mesos.md
@@ -167,8 +167,8 @@ For example:
./bin/spark-submit \
--class org.apache.spark.examples.SparkPi \
--master mesos://207.184.161.138:7077 \
- --deploy-mode cluster
- --supervise
+ --deploy-mode cluster \
+ --supervise \
--executor-memory 20G \
--total-executor-cores 100 \
http://path/to/examples.jar \
http://git-wip-us.apache.org/repos/asf/spark/blob/c11ea2e4/docs/running-on-yarn.md
----------------------------------------------------------------------
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index 8045f8c..c775fe7 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -49,8 +49,8 @@ In `cluster` mode, the driver runs on a different machine than the client, so `S
$ ./bin/spark-submit --class my.main.Class \
--master yarn \
--deploy-mode cluster \
- --jars my-other-jar.jar,my-other-other-jar.jar
- my-main-jar.jar
+ --jars my-other-jar.jar,my-other-other-jar.jar \
+ my-main-jar.jar \
app_arg1 app_arg2
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org