svn commit: r1818150 - /knox/trunk/books/0.14.0/service_livy.md

[lm...@apache.org]

Author: lmccay
Date: Thu Dec 14 15:39:51 2017
New Revision: 1818150

URL: http://svn.apache.org/viewvc?rev=1818150&view=rev
Log:
Update site for 0.14.0 release

Added:
    knox/trunk/books/0.14.0/service_livy.md

Added: knox/trunk/books/0.14.0/service_livy.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/service_livy.md?rev=1818150&view=auto
==============================================================================
--- knox/trunk/books/0.14.0/service_livy.md (added)
+++ knox/trunk/books/0.14.0/service_livy.md Thu Dec 14 15:39:51 2017
@@ -0,0 +1,53 @@
+<!---
+   Licensed to the Apache Software Foundation (ASF) under one or more
+   contributor license agreements.  See the NOTICE file distributed with
+   this work for additional information regarding copyright ownership.
+   The ASF licenses this file to You under the Apache License, Version 2.0
+   (the "License"); you may not use this file except in compliance with
+   the License.  You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+--->
+
+### Livy Server ###
+
+Knox provides proxied access to Livy server for submitting Spark jobs.
+The gateway can be used to provide authentication and encryption for clients to
+servers like Livy.
+
+#### Gateway configuration ####
+
+The Gateway can be configured for Livy by modifying the topology XML file
+and providing a new service XML file.
+
+In the topology XML file, add the following with the correct hostname:
+
+    <service>
+      <role>LIVYSERVER</role>
+      <url>http://<livy-server>:8998</url>
+    </service>
+
+Livy server will use proxUser to run the Spark session. To avoid that a user can 
+provide here any user (e.g. a more privileged), Knox will need to rewrite the the 
+json body to replace what so ever is the value of proxyUser is with the username of
+the authenticated user.
+
+   {  
+      "driverMemory":"2G",
+      "executorCores":4,
+      "executorMemory":"8G",
+      "proxyUser":"bernhard",
+      "conf":{  
+         "spark.master":"yarn-cluster",
+         "spark.jars.packages":"com.databricks:spark-csv_2.10:1.5.0"
+         }
+    } 
+
+The above is an example request body to be used to create a Spark session via Live server and illustrates the "proxyUser" that requires rewrite.
+