You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mo...@apache.org on 2016/05/06 17:03:06 UTC
incubator-zeppelin git commit: Improve README.md
Repository: incubator-zeppelin
Updated Branches:
refs/heads/master be0871490 -> 664fa421d
Improve README.md
### What is this PR for?
Update README.md
- Unify code block styles (fenced code blocks).
- Add syntax highlighting in some code blocks.
- Fix unwanted link in option: `-Pmapr[version]`.
- General formatting.
### What type of PR is it?
Documentation
### Todos
None.
### What is the Jira issue?
N/A
### How should this be tested?
Preview the rendered README.md on GitHub.
View `README.md` on [cyhsutw/incubator-zeppelin:better-readme](https://github.com/cyhsutw/incubator-zeppelin/tree/better-readme)
### Screenshots (if appropriate)
Unwanted link in option: `-Pmapr[version]`:
### Questions:
* Does the licenses files need update? *no*
* Is there breaking changes for older versions? *no*
* Does this needs documentation? *no*
Author: Cheng-Yu Hsu <m...@cyhsu.me>
Closes #839 from cyhsutw/better-readme and squashes the following commits:
93f5c87 [Cheng-Yu Hsu] cleanup whitespace
ff8f7f2 [Cheng-Yu Hsu] clarity the description of `-Pr` and `-Psparkr`
facdf66 [Cheng-Yu Hsu] add `-Pr` and `-Psparkr`
127955b [Cheng-Yu Hsu] update README.md : add `-Psparkr` option
aee2f43 [Cheng-Yu Hsu] Update README.md
Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/664fa421
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/664fa421
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/664fa421
Branch: refs/heads/master
Commit: 664fa421d91c41546a73e660893437c525961ce0
Parents: be08714
Author: Cheng-Yu Hsu <m...@cyhsu.me>
Authored: Sat Apr 30 11:43:25 2016 +0800
Committer: Lee moon soo <mo...@apache.org>
Committed: Fri May 6 10:03:31 2016 -0700
----------------------------------------------------------------------
README.md | 106 ++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 63 insertions(+), 43 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/664fa421/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 20b2669..4265104 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-#Zeppelin
+#Zeppelin
**Documentation:** [User Guide](http://zeppelin.incubator.apache.org/docs/latest/index.html)<br/>
**Mailing Lists:** [User and Dev mailing list](http://zeppelin.incubator.apache.org/community.html)<br/>
@@ -21,15 +21,15 @@ To know more about Zeppelin, visit our web site [http://zeppelin.incubator.apach
* Java 1.7
* Tested on Mac OSX, Ubuntu 14.X, CentOS 6.X
* Maven (if you want to build from the source code)
- * Node.js Package Manager
+ * Node.js Package Manager (npm)
## Getting Started
### Before Build
-If you don't have requirements prepared, install it.
+If you don't have requirements prepared, install it.
(The installation method may vary according to your environment, example is for Ubuntu.)
-```
+```sh
sudo apt-get update
sudo apt-get install git
sudo apt-get install openjdk-7-jdk
@@ -42,10 +42,10 @@ sudo tar -zxf apache-maven-3.3.3-bin.tar.gz -C /usr/local/
sudo ln -s /usr/local/apache-maven-3.3.3/bin/mvn /usr/local/bin/mvn
```
-_Notes:_
+_Notes:_
- Ensure node is installed by running `node --version`
- Ensure maven is running version 3.1.x or higher with `mvn -version`
- - Configure maven to use more memory than usual by ```export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=1024m"```
+ - Configure maven to use more memory than usual by `export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=1024m"`
### Build
If you want to build Zeppelin from the source, please first clone this repository, then:
@@ -61,7 +61,7 @@ Each Interpreter requires different Options.
To build with a specific Spark version, Hadoop version or specific features, define one or more of the following profiles and options:
-##### -Pspark-[version]
+##### `-Pspark-[version]`
Set spark major version
@@ -84,7 +84,7 @@ Available profiles are
minor version can be adjusted by `-Dspark.version=x.x.x`
-##### -Phadoop-[version]
+##### `-Phadoop-[version]`
set hadoop major version
@@ -101,25 +101,32 @@ Available profiles are
minor version can be adjusted by `-Dhadoop.version=x.x.x`
-##### -Pyarn (optional)
+##### `-Pyarn` (optional)
enable YARN support for local mode
-> YARN for local mode is not supported for Spark v1.5.0 or higher. Set SPARK_HOME instead.
+> YARN for local mode is not supported for Spark v1.5.0 or higher. Set `SPARK_HOME` instead.
+
+##### `-Ppyspark` (optional)
+
+enable [PySpark](http://spark.apache.org/docs/latest/api/python/) support for local mode.
-##### -Ppyspark (optional)
+##### `-Pr` (optional)
-enable PySpark support for local mode
+enable [R](https://www.r-project.org/) support with [SparkR](https://spark.apache.org/docs/latest/sparkr.html) integration.
+##### `-Psparkr` (optional)
-##### -Pvendor-repo (optional)
+another [R](https://www.r-project.org/) support with [SparkR](https://spark.apache.org/docs/latest/sparkr.html) integration as well as local mode support.
+
+##### `-Pvendor-repo` (optional)
enable 3rd party vendor repository (cloudera)
-##### -Pmapr[version] (optional)
+##### `-Pmapr[version]` (optional)
-For the MapR Hadoop Distribution, these profiles will handle the Hadoop version. As MapR allows different versions
-of Spark to be installed, you should specify which version of Spark is installed on the cluster by adding a Spark profile (-Pspark-1.2, -Pspark-1.3, etc.) as needed. For Hive, check the hive/pom.xml and adjust the version installed as well. The correct Maven
+For the MapR Hadoop Distribution, these profiles will handle the Hadoop version. As MapR allows different versions of Spark to be installed, you should specify which version of Spark is installed on the cluster by adding a Spark profile (`-Pspark-1.2`, `-Pspark-1.3`, etc.) as needed.
+For Hive, check the hive/pom.xml and adjust the version installed as well. The correct Maven
artifacts can be found for every version of MapR at http://doc.mapr.com
Available profiles are
@@ -135,7 +142,7 @@ Available profiles are
Here're some examples:
-```
+```sh
# basic build
mvn clean package -Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark
@@ -152,13 +159,13 @@ mvn clean package -Pspark-1.5 -Pmapr50 -DskipTests
#### Ignite Interpreter
-```
+```sh
mvn clean package -Dignite.version=1.1.0-incubating -DskipTests
```
#### Scalding Interpreter
-```
+```sh
mvn clean package -Pscalding -DskipTests
```
@@ -169,67 +176,80 @@ If you wish to configure Zeppelin option (like port number), configure the follo
./conf/zeppelin-env.sh
./conf/zeppelin-site.xml
```
-(You can copy ```./conf/zeppelin-env.sh.template``` into ```./conf/zeppelin-env.sh```.
-Same for ```zeppelin-site.xml```.)
+
+(You can copy `./conf/zeppelin-env.sh.template` into `./conf/zeppelin-env.sh`.
+Same for `zeppelin-site.xml`.)
#### Setting SPARK_HOME and HADOOP_HOME
-Without SPARK_HOME and HADOOP_HOME, Zeppelin uses embedded Spark and Hadoop binaries that you have specified with mvn build option.
-If you want to use system provided Spark and Hadoop, export SPARK_HOME and HADOOP_HOME in zeppelin-env.sh
+Without `SPARK_HOME` and `HADOOP_HOME`, Zeppelin uses embedded Spark and Hadoop binaries that you have specified with mvn build option.
+If you want to use system provided Spark and Hadoop, export `SPARK_HOME` and `HADOOP_HOME` in `zeppelin-env.sh`.
You can use any supported version of spark without rebuilding Zeppelin.
-```
+```sh
# ./conf/zeppelin-env.sh
export SPARK_HOME=...
export HADOOP_HOME=...
```
#### External cluster configuration
+
Mesos
- # ./conf/zeppelin-env.sh
- export MASTER=mesos://...
- export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=/path/to/spark-*.tgz" or SPARK_HOME="/path/to/spark_home"
- export MESOS_NATIVE_LIBRARY=/path/to/libmesos.so
-
+```sh
+# ./conf/zeppelin-env.sh
+export MASTER=mesos://...
+export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=/path/to/spark-*.tgz" or SPARK_HOME="/path/to/spark_home"
+export MESOS_NATIVE_LIBRARY=/path/to/libmesos.so
+```
+
If you set `SPARK_HOME`, you should deploy spark binary on the same location to all worker nodes. And if you set `spark.executor.uri`, every worker can read that file on its node.
Yarn
- # ./conf/zeppelin-env.sh
- export SPARK_HOME=/path/to/spark_dir
+```sh
+# ./conf/zeppelin-env.sh
+export SPARK_HOME=/path/to/spark_dir
+```
### Run
- ./bin/zeppelin-daemon.sh start
- browse localhost:8080 in your browser.
+```sh
+./bin/zeppelin-daemon.sh start
+```
+
+And browse [localhost:8080](localhost:8080) in your browser.
-For configuration details check __./conf__ subdirectory.
+For configuration details check __`./conf`__ subdirectory.
### Package
To package the final distribution including the compressed archive, run:
- mvn clean package -Pbuild-distr
+```sh
+mvn clean package -Pbuild-distr
+```
To build a distribution with specific profiles, run:
- mvn clean package -Pbuild-distr -Pspark-1.5 -Phadoop-2.4 -Pyarn -Ppyspark
+```sh
+mvn clean package -Pbuild-distr -Pspark-1.5 -Phadoop-2.4 -Pyarn -Ppyspark
+```
The profiles `-Pspark-1.5 -Phadoop-2.4 -Pyarn -Ppyspark` can be adjusted if you wish to build to a specific spark versions, or omit support such as `yarn`.
-The archive is generated under _zeppelin-distribution/target_ directory
+The archive is generated under _`zeppelin-distribution/target`_ directory
###Run end-to-end tests
Zeppelin comes with a set of end-to-end acceptance tests driving headless selenium browser
- #assumes zeppelin-server running on localhost:8080 (use -Durl=.. to override)
- mvn verify
-
- #or take care of starting\stoping zeppelin-server from packaged _zeppelin-distribuion/target_
- mvn verify -P using-packaged-distr
-
+```sh
+# assumes zeppelin-server running on localhost:8080 (use -Durl=.. to override)
+mvn verify
+# or take care of starting/stoping zeppelin-server from packaged zeppelin-distribuion/target
+mvn verify -P using-packaged-distr
+```
[](https://github.com/igrigorik/ga-beacon)