[2/4] kylin git commit: Add spark installation in dev env setup doc

[sh...@apache.org]

Add spark installation in dev env setup doc


Project: http://git-wip-us.apache.org/repos/asf/kylin/repo
Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/583c940c
Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/583c940c
Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/583c940c

Branch: refs/heads/document
Commit: 583c940c21473fc445c100853d811628b3e5e0a6
Parents: cb70936
Author: shaofengshi <sh...@apache.org>
Authored: Wed Mar 29 16:19:52 2017 +0800
Committer: shaofengshi <sh...@apache.org>
Committed: Wed Mar 29 16:19:52 2017 +0800

----------------------------------------------------------------------
 website/_dev/dev_env.md | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kylin/blob/583c940c/website/_dev/dev_env.md
----------------------------------------------------------------------
diff --git a/website/_dev/dev_env.md b/website/_dev/dev_env.md
index 1147265..c16330a 100644
--- a/website/_dev/dev_env.md
+++ b/website/_dev/dev_env.md
@@ -31,7 +31,7 @@ For other hadoop distribution, basically start the hadoop cluster, make sure HDF
 ## Environment on the dev machine
 
 
-### Install maven
+### Install Maven
 
 The latest maven can be found at <http://maven.apache.org/download.cgi>, we create a symbolic so that `mvn` can be run anywhere.
 
@@ -42,6 +42,31 @@ tar -xzvf apache-maven-3.2.5-bin.tar.gz
 ln -s /root/apache-maven-3.2.5/bin/mvn /usr/bin/mvn
 {% endhighlight %}
 
+### Install Spark
+
+Manually install spark-1.6.3-bin-hadoop2.6 in a local folder like /usr/local/spark
+
+{% highlight Groff markup %}
+wget -O /tmp/spark-1.6.3-bin-hadoop2.6.tgz http://d3kbcqa49mib13.cloudfront.net/spark-1.6.3-bin-hadoop2.6.tgz
+cd /usr/local
+tar -zxvf /tmp/spark-1.6.3-bin-hadoop2.6.tgz
+ln -s spark-1.6.3-bin-hadoop2.6 spark
+{% endhighlight %}
+
+Upload the spark-assembly jar to HDFS as /kylin/spark/spark-assembly-1.6.3-hadoop2.6.0.jar (avoid repeatedly uploading the jar to HDFS):
+
+{% highlight Groff markup %}
+hadoop fs -mkdir /kylin/spark/
+hadoop fs -put /usr/local/spark/lib/spark-assembly-1.6.3-hadoop2.6.0.jar /kylin/spark/
+{% endhighlight %}
+
+Create local temp folder for hbase client (if it doesn't exist):
+
+{% highlight Groff markup %}
+mkdir -p /hadoop/hbase/local/jars
+chmod 777 /hadoop/hbase/local/jars
+{% endhighlight %}
+
 ### Compile
 
 First clone the Kylin project to your local: