You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mi...@apache.org on 2016/02/28 07:21:14 UTC
svn commit: r1732691 - in
/incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT: atom.xml
install/install.html install/yarn_install.html rss.xml
Author: minalee
Date: Sun Feb 28 06:21:14 2016
New Revision: 1732691
URL: http://svn.apache.org/viewvc?rev=1732691&view=rev
Log:
https://github.com/apache/incubator-zeppelin/pull/736
Modified:
incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/atom.xml
incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/install.html
incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/yarn_install.html
incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/rss.xml
Modified: incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/atom.xml
URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/atom.xml?rev=1732691&r1=1732690&r2=1732691&view=diff
==============================================================================
--- incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/atom.xml (original)
+++ incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/atom.xml Sun Feb 28 06:21:14 2016
@@ -4,7 +4,7 @@
<title>Apache Zeppelin (incubating)</title>
<link href="http://zeppelin.incubator.apache.org/" rel="self"/>
<link href="http://zeppelin.incubator.apache.org"/>
- <updated>2016-02-20T08:54:53-08:00</updated>
+ <updated>2016-02-28T15:15:24+09:00</updated>
<id>http://zeppelin.incubator.apache.org</id>
<author>
<name>The Apache Software Foundation</name>
Modified: incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/install.html
URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/install.html?rev=1732691&r1=1732690&r2=1732691&view=diff
==============================================================================
--- incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/install.html (original)
+++ incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/install.html Sun Feb 28 06:21:14 2016
@@ -166,9 +166,9 @@ limitations under the License.
<h2>Zeppelin Installation</h2>
-<p>Welcome to your first trial to explore Zeppelin ! </p>
+<p>Welcome to your first trial to explore Zeppelin!</p>
-<p>In this documentation, we will explain how you can install Zeppelin from <strong>Binary Package</strong> or build from <strong>Source</strong> by yourself. Plus, you can see all of Zeppelin's configurations in the <strong>Zeppelin Configuration</strong> section below.</p>
+<p>In this documentation, we will explain how you can install Zeppelin from <strong>Binary Package</strong> or build from <strong>Source</strong> by yourself. Plus, you can see all of Zeppelin's configurations in the <a href="install.html#zeppelin-configuration">Zeppelin Configuration</a> section below.</p>
<h3>Install with Binary Package</h3>
@@ -176,7 +176,20 @@ limitations under the License.
<h3>Build from Zeppelin Source</h3>
-<p>You can also build Zeppelin from the source. Please check instructions in <code>README.md</code> in <a href="https://github.com/apache/incubator-zeppelin/blob/master/README.md">Zeppelin github</a>. </p>
+<p>You can also build Zeppelin from the source.</p>
+
+<h4>Prerequisites for build</h4>
+
+<ul>
+<li>Java 1.7</li>
+<li>Git</li>
+<li>Maven(3.1.x or higher)</li>
+<li>Node.js Package Manager</li>
+</ul>
+
+<p>If you don't have requirements prepared, please check instructions in <a href="https://github.com/apache/incubator-zeppelin/blob/master/README.md">README.md</a> for the details.</p>
+
+<p><a name="zeppelin-configuration"> </a></p>
<h2>Zeppelin Configuration</h2>
@@ -375,6 +388,37 @@ limitations under the License.
<h4>Stop Zeppelin</h4>
<div class="highlight"><pre><code class="text language-text" data-lang="text">bin/zeppelin-daemon.sh stop
</code></pre></div>
+<h4>Start Zeppelin with a service manager such as upstart</h4>
+
+<p>Zeppelin can auto start as a service with an init script, such as services managed by upstart.</p>
+
+<p>The following is an example upstart script to be saved as <code>/etc/init/zeppelin.conf</code>
+This example has been tested with Ubuntu Linux.
+This also allows the service to be managed with commands such as </p>
+
+<p><code>sudo service zeppelin start</code><br>
+<code>sudo service zeppelin stop</code><br>
+<code>sudo service zeppelin restart</code></p>
+
+<p>Other service managers could use a similar approach with the <code>upstart</code> argument passed to the zeppelin-daemon.sh script: <code>bin/zeppelin-daemon.sh upstart</code></p>
+
+<h5>zeppelin.conf</h5>
+<div class="highlight"><pre><code class="text language-text" data-lang="text">description "zeppelin"
+
+start on (local-filesystems and net-device-up IFACE!=lo)
+stop on shutdown
+
+# Respawn the process on unexpected termination
+respawn
+
+# respawn the job up to 7 times within a 5 second period.
+# If the job exceeds these values, it will be stopped and marked as failed.
+respawn limit 7 5
+
+# zeppelin was installed in /usr/share/zeppelin in this example
+chdir /usr/share/zeppelin
+exec bin/zeppelin-daemon.sh upstart
+</code></pre></div>
</div>
</div>
Modified: incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/yarn_install.html
URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/yarn_install.html?rev=1732691&r1=1732690&r2=1732691&view=diff
==============================================================================
--- incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/yarn_install.html (original)
+++ incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/install/yarn_install.html Sun Feb 28 06:21:14 2016
@@ -166,7 +166,7 @@ limitations under the License.
<h2>Introduction</h2>
-<p>This page describes how to pre-configure a bare metal node, build & configure Zeppelin on it, configure Zeppelin and connect it to existing YARN cluster running Hortonworks flavour of Hadoop. It also describes steps to configure Spark & Hive interpreter of Zeppelin. </p>
+<p>This page describes how to pre-configure a bare metal node, configure Zeppelin and connect it to existing YARN cluster running Hortonworks flavour of Hadoop. It also describes steps to configure Spark & Hive interpreter of Zeppelin.</p>
<h2>Prepare Node</h2>
@@ -187,72 +187,18 @@ whoami
<h3>List of Prerequisites</h3>
<ul>
-<li>CentOS 6.x</li>
-<li>Git</li>
-<li>Java 1.7 </li>
-<li>Apache Maven</li>
-<li>Hadoop client.</li>
-<li>Spark.</li>
+<li>CentOS 6.x, Mac OSX, Ubuntu 14.X</li>
+<li>Java 1.7</li>
+<li>Hadoop client</li>
+<li>Spark</li>
<li>Internet connection is required. </li>
</ul>
-<p>Its assumed that the node has CentOS 6.x installed on it. Although any version of Linux distribution should work fine. The working directory of all prerequisite pacakges is /home/zeppelin/prerequisites, although any location could be used.</p>
+<p>It's assumed that the node has CentOS 6.x installed on it. Although any version of Linux distribution should work fine.</p>
-<h4>Git</h4>
-
-<p>Intall latest stable version of Git. This document describes installation of version 2.4.8</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">yum install curl-devel expat-devel gettext-devel openssl-devel zlib-devel
-yum install gcc perl-ExtUtils-MakeMaker
-yum remove git
-<span class="nb">cd</span> /home/zeppelin/prerequisites
-wget https://github.com/git/git/archive/v2.4.8.tar.gz
-tar xzf git-2.0.4.tar.gz
-<span class="nb">cd </span>git-2.0.4
-make <span class="nv">prefix</span><span class="o">=</span>/home/zeppelin/prerequisites/git all
-make <span class="nv">prefix</span><span class="o">=</span>/home/zeppelin/prerequisites/git install
-<span class="nb">echo</span> <span class="s2">"export PATH=$PATH:/home/zeppelin/prerequisites/bin"</span> >> /home/zeppelin/.bashrc
-<span class="nb">source</span> /home/zeppelin/.bashrc
-git --version
-</code></pre></div>
-<p>Assuming all the packages are successfully installed, running the version option with git command should display</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">git version 2.4.8
-</code></pre></div>
-<h4>Java</h4>
-
-<p>Zeppelin works well with 1.7.x version of Java runtime. Download JDK version 7 and a stable update and follow below instructions to install it.</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">cd</span> /home/zeppelin/prerequisites/
-<span class="c">#Download JDK 1.7, Assume JDK 7 update 79 is downloaded.</span>
-tar -xf jdk-7u79-linux-x64.tar.gz
-<span class="nb">echo</span> <span class="s2">"export JAVA_HOME=/home/zeppelin/prerequisites/jdk1.7.0_79"</span> >> /home/zeppelin/.bashrc
-<span class="nb">source</span> /home/zeppelin/.bashrc
-<span class="nb">echo</span> <span class="nv">$JAVA_HOME</span>
-</code></pre></div>
-<p>Assuming all the packages are successfully installed, echoing JAVA_HOME environment variable should display</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">/home/zeppelin/prerequisites/jdk1.7.0_79
-</code></pre></div>
-<h4>Apache Maven</h4>
-
-<p>Download and install a stable version of Maven.</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">cd</span> /home/zeppelin/prerequisites/
-wget ftp://mirror.reverse.net/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
-tar -xf apache-maven-3.3.3-bin.tar.gz
-<span class="nb">cd </span>apache-maven-3.3.3
-<span class="nb">export </span><span class="nv">MAVEN_HOME</span><span class="o">=</span>/home/zeppelin/prerequisites/apache-maven-3.3.3
-<span class="nb">echo</span> <span class="s2">"export PATH=$PATH:/home/zeppelin/prerequisites/apache-maven-3.3.3/bin"</span> >> /home/zeppelin/.bashrc
-<span class="nb">source</span> /home/zeppelin/.bashrc
-mvn -version
-</code></pre></div>
-<p>Assuming all the packages are successfully installed, running the version option with mvn command should display</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">Apache Maven 3.3.3 <span class="o">(</span>7994120775791599e205a5524ec3e0dfe41d4a06<span class="p">;</span> 2015-04-22T04:57:37-07:00<span class="o">)</span>
-Maven home: /home/zeppelin/prerequisites/apache-maven-3.3.3
-Java version: 1.7.0_79, vendor: Oracle Corporation
-Java home: /home/zeppelin/prerequisites/jdk1.7.0_79/jre
-Default locale: en_US, platform encoding: UTF-8
-OS name: <span class="s2">"linux"</span>, version: <span class="s2">"2.6.32-358.el6.x86_64"</span>, arch: <span class="s2">"amd64"</span>, family: <span class="s2">"unix"</span>
-</code></pre></div>
<h4>Hadoop client</h4>
-<p>Zeppelin can work with multiple versions & distributions of Hadoop. A complete list <a href="https://github.com/apache/incubator-zeppelin#build">is available here.</a> This document assumes Hadoop 2.7.x client libraries including configuration files are installed on Zeppelin node. It also assumes /etc/hadoop/conf contains various Hadoop configuration files. The location of Hadoop configuration files may vary, hence use appropriate location.</p>
+<p>Zeppelin can work with multiple versions & distributions of Hadoop. A complete list is available <a href="https://github.com/apache/incubator-zeppelin#build">here</a>. This document assumes Hadoop 2.7.x client libraries including configuration files are installed on Zeppelin node. It also assumes /etc/hadoop/conf contains various Hadoop configuration files. The location of Hadoop configuration files may vary, hence use appropriate location.</p>
<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">hadoop version
Hadoop 2.7.1.2.3.1.0-2574
Subversion git@github.com:hortonworks/hadoop.git -r f66cf95e2e9367a74b0ec88b2df33458b6cff2d0
@@ -263,35 +209,32 @@ This <span class="nb">command </span>was
</code></pre></div>
<h4>Spark</h4>
-<p>Zeppelin can work with multiple versions Spark. A complete list <a href="https://github.com/apache/incubator-zeppelin#build">is available here.</a> This document assumes Spark 1.3.1 is installed on Zeppelin node at /home/zeppelin/prerequisites/spark.</p>
-
-<h2>Build</h2>
+<p>Spark is supported out of the box and to take advantage of this, you need to Download appropriate version of Spark binary packages from <a href="http://spark.apache.org/downloads.html">Spark Download page</a> and unzip it.
+Zeppelin can work with multiple versions of Spark. A complete list is available <a href="https://github.com/apache/incubator-zeppelin#build">here</a>.
+This document assumes Spark 1.6.0 is installed at /usr/lib/spark.</p>
-<p>Checkout source code from <a href="git://git.apache.org/incubator-zeppelin.git">git://git.apache.org/incubator-zeppelin.git</a>.</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">cd</span> /home/zeppelin/
-git clone git://git.apache.org/incubator-zeppelin.git
-</code></pre></div>
-<p>Zeppelin package is available at <code>/home/zeppelin/incubator-zeppelin</code> after the checkout completes.</p>
+<blockquote>
+<p>Note: Spark should be installed on the same node as Zeppelin.</p>
-<h3>Cluster mode</h3>
+<p>Note: Spark's pre-built package for CDH 4 doesn't support yarn.</p>
+</blockquote>
-<p>As its assumed Hadoop 2.7.x is installed on the YARN cluster & Spark 1.3.1 is installed on Zeppelin node. Hence appropriate options are chosen to build Zeppelin. This is very important as Zeppelin will bundle corresponding Hadoop & Spark libraries and they must match the ones present on YARN cluster & Zeppelin Spark installation. </p>
+<h4>Zeppelin</h4>
-<p>Zeppelin is a maven project and hence must be built with Apache Maven.</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">cd</span> /home/zeppelin/incubator-zeppelin
-mvn clean package -Pspark-1.3 -Dspark.version<span class="o">=</span>1.3.1 -Dhadoop.version<span class="o">=</span>2.7.0 -Phadoop-2.6 -Pyarn -DskipTests
-</code></pre></div>
-<p>Building Zeppelin for first time downloads various dependencies and hence takes few minutes to complete. </p>
+<p>Checkout source code from <a href="https://github.com/apache/incubator-zeppelin.git">git://git.apache.org/incubator-zeppelin.git</a> or download binary package from <a href="https://zeppelin.incubator.apache.org/download.html">Download page</a>.
+You can refer <a href="install.html">Install</a> page for the details.
+This document assumes that Zeppelin is located under <code>/home/zeppelin/incubator-zeppelin</code>.</p>
<h2>Zeppelin Configuration</h2>
-<p>Zeppelin configurations needs to be modified to connect to YARN cluster. Create a copy of zeppelin environment XML</p>
+<p>Zeppelin configuration needs to be modified to connect to YARN cluster. Create a copy of zeppelin environment shell script.</p>
<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">cp /home/zeppelin/incubator-zeppelin/conf/zeppelin-env.sh.template /home/zeppelin/incubator-zeppelin/conf/zeppelin-env.sh
</code></pre></div>
<p>Set the following properties</p>
-<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span>/home/zeppelin/prerequisites/jdk1.7.0_79
-<span class="nb">export </span><span class="nv">HADOOP_CONF_DIR</span><span class="o">=</span>/etc/hadoop/conf
+<div class="highlight"><pre><code class="bash language-bash" data-lang="bash"><span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span><span class="s2">"/usr/java/jdk1.7.0_79"</span>
+<span class="nb">export </span><span class="nv">HADOOP_CONF_DIR</span><span class="o">=</span><span class="s2">"/etc/hadoop/conf"</span>
<span class="nb">export </span><span class="nv">ZEPPELIN_JAVA_OPTS</span><span class="o">=</span><span class="s2">"-Dhdp.version=2.3.1.0-2574"</span>
+<span class="nb">export </span><span class="nv">SPARK_HOME</span><span class="o">=</span><span class="s2">"/usr/lib/spark"</span>
</code></pre></div>
<p>As /etc/hadoop/conf contains various configurations of YARN cluster, Zeppelin can now submit Spark/Hive jobs on YARN cluster form its web interface. The value of hdp.version is set to 2.3.1.0-2574. This can be obtained by running the following command</p>
<div class="highlight"><pre><code class="bash language-bash" data-lang="bash">hdp-select status hadoop-client <span class="p">|</span> sed <span class="s1">'s/hadoop-client - \(.*\)/\1/'</span>
@@ -310,7 +253,7 @@ bin/zeppelin-daemon.sh start
</code></pre></div>
<h2>Interpreter</h2>
-<p>Zeppelin provides to various distributed processing frameworks to process data that ranges from Spark, Hive, Tajo, Ignite and Lens to name a few. This document describes to configure Hive & Spark interpreters.</p>
+<p>Zeppelin provides various distributed processing frameworks to process data that ranges from Spark, Hive, Tajo, Ignite and Lens to name a few. This document describes to configure Hive & Spark interpreters.</p>
<h3>Hive</h3>
@@ -322,7 +265,7 @@ Click on Save button. Once these configu
<h3>Spark</h3>
-<p>Zeppelin was built with Spark 1.3.1 and it was assumed that 1.3.1 version of Spark is installed at /home/zeppelin/prerequisites/spark. Look for Spark configrations and click edit button to add the following properties</p>
+<p>It was assumed that 1.6.0 version of Spark is installed at /usr/lib/spark. Look for Spark configurations and click edit button to add the following properties</p>
<table class="table-configuration">
<tr>
@@ -336,11 +279,6 @@ Click on Save button. Once these configu
<td>In yarn-client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN.</td>
</tr>
<tr>
- <td>spark.home</td>
- <td>/home/zeppelin/prerequisites/spark</td>
- <td></td>
- </tr>
- <tr>
<td>spark.driver.extraJavaOptions</td>
<td>-Dhdp.version=2.3.1.0-2574</td>
<td></td>
@@ -350,11 +288,6 @@ Click on Save button. Once these configu
<td>-Dhdp.version=2.3.1.0-2574</td>
<td></td>
</tr>
- <tr>
- <td>spark.yarn.jar</td>
- <td>/home/zeppelin/incubator-zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar</td>
- <td></td>
- </tr>
</table>
<p>Click on Save button. Once these configurations are updated, Zeppelin will prompt you to restart the interpreter. Accept the prompt and the interpreter will reload the configurations.</p>
Modified: incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/rss.xml
URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/rss.xml?rev=1732691&r1=1732690&r2=1732691&view=diff
==============================================================================
--- incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/rss.xml (original)
+++ incubator/zeppelin/site/docs/0.6.0-incubating-SNAPSHOT/rss.xml Sun Feb 28 06:21:14 2016
@@ -5,8 +5,8 @@
<description>Apache Zeppelin (incubating) - The Apache Software Foundation</description>
<link>http://zeppelin.incubator.apache.org</link>
<link>http://zeppelin.incubator.apache.org</link>
- <lastBuildDate>2016-02-20T08:54:53-08:00</lastBuildDate>
- <pubDate>2016-02-20T08:54:53-08:00</pubDate>
+ <lastBuildDate>2016-02-28T15:15:24+09:00</lastBuildDate>
+ <pubDate>2016-02-28T15:15:24+09:00</pubDate>
<ttl>1800</ttl>