You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by su...@apache.org on 2015/05/04 20:52:17 UTC
svn commit: r1677654 - in /knox: site/ site/books/knox-0-6-0/
trunk/books/0.6.0/ trunk/books/0.6.0/dev-guide/
Author: sumit
Date: Mon May 4 18:52:17 2015
New Revision: 1677654
URL: http://svn.apache.org/r1677654
Log:
KNOX-481 docs for Stacks and misc fixes
Modified:
knox/site/books/knox-0-6-0/dev-guide.html
knox/site/books/knox-0-6-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.6.0/book.md
knox/trunk/books/0.6.0/book_service-details.md
knox/trunk/books/0.6.0/dev-guide/book.md
knox/trunk/books/0.6.0/service_storm.md
Modified: knox/site/books/knox-0-6-0/dev-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/dev-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/dev-guide.html (original)
+++ knox/site/books/knox-0-6-0/dev-guide.html Mon May 4 18:52:17 2015
@@ -144,6 +144,10 @@
<td>The implementation of the Access service POC. </td>
</tr>
<tr>
+ <td>gateway-service-definitions </td>
+ <td>The implementation of the Service definition and rewrite files. </td>
+ </tr>
+ <tr>
<td>gateway-service-hbase </td>
<td>The implementation of the HBase service. </td>
</tr>
@@ -160,10 +164,6 @@
<td>The implementation of the Ticket Granting service POC. </td>
</tr>
<tr>
- <td>gateway-service-webhcat </td>
- <td>The implementation of the WebHCat service. </td>
- </tr>
- <tr>
<td>gateway-service-webhdfs </td>
<td>The implementation of the WebHdfs service. </td>
</tr>
@@ -521,6 +521,94 @@ public interface ServiceDeploymentContri
</dependencies>
</project>
+</code></pre><h4><a id="Service+Definition+Files"></a>Service Definition Files</h4><p>As of release 0.6.0, the gateway now also supports a declarative way of plugging-in a new Service. A Service can be defined with a combination of two files, these are:</p>
+<pre><code>service.xml
+rewrite.xml
+</code></pre><p>The rewrite.xml file contains the rewrite rules as defined in other sections of this guide, and the service.xml file contains the various routes (paths) to be provided by the Service and the rewrite rule bindings to those paths. This will be described in further detail in this section.</p><p>While the service.xml file is absolutely required, the rewrite.xml file in theory is optional (though it is highly unlikely that no rewrite rules are needed).</p><p>To add a new service, simply add a service.xml and rewrite.xml file in an appropriate directory (see <a href="#Service+Definition+Directory+Structure">Service Definition Directory Structure</a>) in the module gateway-service-definitions to make the new service part of the Knox build.</p><h5><a id="service.xml"></a>service.xml</h5><p>Below is a sample of a very simple service.xml file, taking the same weather api example.</p>
+<pre><code class="xml"><service role="WEATHER" name="weather" version="0.1.0">
+ <routes>
+ <route path="/weather/**?**"/>
+ </routes>
+</service>
+
+</code></pre>
+<dl><dt><strong>service</strong></dt><dd>The root tag is ‘service’ that has the three required attributes: <em>role</em>, <em>name</em> and <em>version</em>. These three values disambiguate this service definition from others. To ensure the exact same service definition is being used in a topology file, all values should be specified,</dd>
+</dl>
+<pre><code class="xml"><topology>
+ <gateway>
+ ....
+ </gateway>
+ <service>
+ <role>WEATHER</role>
+ <name>weather</name>
+ <version>0.1.0</version>
+ <url>http://api.openweathermap.org/data</url>
+ </service>
+ ....
+</topology>
+</code></pre><p>If only <em>role</em> is specified in the topology file (the only required element other than <em>url</em>) then the first service definition of that role found will be used with the highest version of that role and name. Similarly if only the <em>version</em> is omitted from the topology specification of the service, the service definition of the highest version will be used. It is therefore important to specify a version for a service if it is desired that a topology be locked down to a specific version of a service.</p>
+<dl><dt><strong>routes</strong></dt><dd>Wrapper element for one or more routes.</dd><dt><strong>route</strong></dt><dd>A route specifies the <em>path</em> that the service is routing as well as any rewrite bindings or policy bindings. Another child element that may be used here is a <em>dispatch</em> element.</dd><dt><strong>rewrite</strong></dt><dd>A rewrite rule or function that is to be applied to the path. A rewrite element contains a <em>apply</em> attribute that references the rewrite function or rule by name. Along with the <em>apply</em> attribute, a <em>to</em> attribute must be used. The <em>to</em> specifies what part of the request or response to rewrite. The valid values for the <em>to</em> attribute are:</dd>
+</dl>
+<ul>
+ <li>request.url</li>
+ <li>request.headers</li>
+ <li>request.cookies</li>
+ <li>request.body</li>
+ <li>response.headers</li>
+ <li>response.cookies</li>
+ <li>response.body</li>
+</ul><p>Below is an example of a snippet from the WebHDFS service definition</p>
+<pre><code class="xml"> <route path="/webhdfs/v1/**?**">
+ <rewrite apply="WEBHDFS/webhdfs/inbound/namenode/file" to="request.url"/>
+ <rewrite apply="WEBHDFS/webhdfs/outbound/namenode/headers" to="response.headers"/>
+ </route>
+</code></pre>
+<dl><dt><strong>dispatch</strong></dt><dd>The dispatch element can be used to plug-in a custom dispatch class. The interface for Dispatch can be found in the module gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.</dd>
+</dl><p>This element can be used at the service level (i.e. as a child of the service tag) or at the route level. A dispatch specified at the route level takes precedence over a dispatch specified at the service level. By default the dispatch used is org.apache.hadoop.gateway.dispatch.DefaultDispatch.</p><p>The dispatch tag has four attributes that can be specified.</p><p><em>contributor-name</em> : This attribute can be used to specify a deployment contributor to be invoked for a custom dispatch.</p><p><em>classname</em> : This attribute can be used to specify a custom dispatch class.</p><p><em>ha-contributor-name</em> : This attribute can be used to specify a deployment contributor to be invoked for custom HA dispatch functionality.</p><p><em>ha-classname</em> : This attribute can be used to specify a custom dispatch class with HA functionality.</p><p>Only one of contributor-name or classname should be specified and one of ha-contributor-name or ha-classname should be specified.</
p><p>If providing a custom dispatch, either a jar should be provided, see <a href="#Class+Path">Class Path</a> or a <a href="#Maven+Module">Maven Module</a> should be created.</p>
+<dl><dt><strong>policies</strong></dt><dd>This is a wrapper tag for <em>policy</em> elements and can be a child of the <em>service</em> tag or the <em>route</em> tag. Once again, just like with dispatch, the route level policies defined override the ones at the service level.</dd>
+</dl><p>This element can contain one or more <em>policy</em> elements. The order of the <em>policy</em> elements is important as that will be the order of execution.</p>
+<dl><dt><strong>policy</strong></dt><dd>At this time the policy element just has two attributes, <em>role</em> and <em>name</em>. These are used to execute a deployment contributor by that role and name. Therefore new policies must be added by using the deployment contributor mechanism.</dd>
+</dl><p>For example,</p>
+<pre><code class="xml"><service role="FOO" name="foo" version="1.0.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication"/>
+ <policy role="rewrite"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/foo/?**">
+ <rewrite apply="FOO/foo/inbound" to="request.url"/>
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="federation"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ <policy role="rewrite"/>
+ </policies>
+ <dispatch contributor-name="http-client" />
+ </route>
+ </routes>
+ <dispatch contributor-name="custom-client" ha-contributor-name="ha-client"/>
+</service>
+</code></pre><h5><a id="rewrite.xml"></a>rewrite.xml</h5><p>The rewrite.xml file that accompanies the service.xml file follows the same rules as described in the section <a href="#Rewrite+Provider">Rewrite Provider</a>.</p><h4><a id="Service+Definition+Directory+Structure"></a>Service Definition Directory Structure</h4><p>On installation of the Knox gateway, the following directory structure can be found under ${GATEWAY_HOME}/data. This is a mirror of the directories and files under the module gateway-service-definitions.</p>
+<pre><code>services
+ |______ service name
+ |______ version
+ |______service.xml
+ |______rewrite.xml
+</code></pre><p>For example,</p>
+<pre><code>services
+ |______ webhdfs
+ |______ 2.4.0
+ |______service.xml
+ |______rewrite.xml
+</code></pre><p>To test out a new service, you can just add the appropriate files (service.xml and rewrite.xml) in a directory under ${GATEWAY_HOME}/data/services. If you want to make the service contribution to the Knox build, they files need to go in the gateway-service-definitions module.</p><h4><a id="Service+Definition+Runtime+Behavior"></a>Service Definition Runtime Behavior</h4><p>The runtime artifacts as well as the behavior does not change whether the service is plugged in via the deployment descriptors or through a service.xml file.</p><h4><a id="Custom+Dispatch+Dependency+Injection"></a>Custom Dispatch Dependency Injection</h4><p>When writing a custom dispatch class, one often needs configuration or gateway services. A lightweight dependency injection system is used that can inject instances of classes or primitives available in the filter configuration’s init params or as a servlet context attribute.</p><p>Details of this can be found in the module gateway-util-con
figinjector and also an example use of it is in the class org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following method for example:</p>
+<pre><code class="java"> @Configure
+ protected void setReplayBufferSize(@Default("8") int size) {
+ replayBufferSize = size;
+ }
</code></pre><h3><a id="Providers"></a>Providers</h3>
<pre><code class="java">public interface ProviderDeploymentContributor {
String getRole();
Modified: knox/site/books/knox-0-6-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/user-guide.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/user-guide.html (original)
+++ knox/site/books/knox-0-6-0/user-guide.html Mon May 4 18:52:17 2015
@@ -50,6 +50,7 @@
<li><a href="#HBase">HBase</a></li>
<li><a href="#Hive">Hive</a></li>
<li><a href="#Yarn">Yarn</a></li>
+ <li><a href="#Storm">Storm</a></li>
</ul></li>
<li><a href="#Limitations">Limitations</a></li>
<li><a href="#Troubleshooting">Troubleshooting</a></li>
@@ -2007,6 +2008,7 @@ dep/commons-codec-1.7.jar
<li><a href="#HBase">HBase</a></li>
<li><a href="#Hive">Hive</a></li>
<li><a href="#Yarn">Yarn</a></li>
+ <li><a href="#Storm">Storm</a></li>
</ul><h3><a id="Assumptions"></a>Assumptions</h3><p>This document assumes a few things about your environment in order to simplify the examples.</p>
<ul>
<li>The JVM is executable as simply java.</li>
@@ -3343,7 +3345,7 @@ The contents of the state-killed.json fi
curl -ikv -u guest:guest-password -H Content-Type:application/json -X PUT -T state-killed.json 'https://localhost:8443/gateway/sandbox/resourcemanager/v1/cluster/apps/{application_id}/state'
-</code></pre><h4><a id="Storm"></a>Storm</h4><p>Storm is a distributed realtime computation system. Storm exposes REST APIs for UI functionality that can be used for retrieving metrics data and configuration information as well as management operations such as starting or stopping topologies.</p><p>The docs for this can be found here</p><p><a href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md">https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To enable this functionality, a topology file needs to have the following configuration:</p>
+</code></pre><h3><a id="Storm"></a>Storm</h3><p>Storm is a distributed realtime computation system. Storm exposes REST APIs for UI functionality that can be used for retrieving metrics data and configuration information as well as management operations such as starting or stopping topologies.</p><p>The docs for this can be found here</p><p><a href="https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md">https://github.com/apache/storm/blob/master/STORM-UI-REST-API.md</a></p><p>To enable this functionality, a topology file needs to have the following configuration:</p>
<pre><code><service>
<role>STORM</role>
<url>http://<hostname>:<port></url>
@@ -3377,7 +3379,7 @@ curl -ikv -u guest:guest-password -H Con
<td><code>http://{storm-logviewer-host}:{storm-logviewer-port}</code> </td>
</tr>
</tbody>
-</table><h4><a id="Storm+Examples+via+cURL"></a>Storm Examples via cURL</h4><p>Some of the various calls that can be made and examples using curl are listed below.</p>
+</table><h4><a id="Storm+Examples"></a>Storm Examples</h4><p>Some of the various calls that can be made and examples using curl are listed below.</p>
<pre><code># 0. Getting cluster configuration
curl -ikv -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/storm/api/v1/cluster/configuration'
Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Mon May 4 18:52:17 2015
@@ -1,5 +1,5 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-01 -->
+<!-- Generated by Apache Maven Doxia Site Renderer 1.6 at 2015-05-04 -->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
@@ -10,7 +10,7 @@
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
- <meta name="Date-Revision-yyyymmdd" content="20150501" />
+ <meta name="Date-Revision-yyyymmdd" content="20150504" />
<meta http-equiv="Content-Language" content="en" />
<script type="text/javascript">var _gaq = _gaq || [];
@@ -57,7 +57,7 @@
<a href="https://cwiki.apache.org/confluence/display/KNOX/Index" class="externalLink" title="Wiki">Wiki</a>
- | <span id="publishDate">Last Published: 2015-05-01</span>
+ | <span id="publishDate">Last Published: 2015-05-04</span>
| <span id="projectVersion">Version: 0.0.0-SNAPSHOT</span>
</div>
<div class="clear">
Modified: knox/trunk/books/0.6.0/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book.md (original)
+++ knox/trunk/books/0.6.0/book.md Mon May 4 18:52:17 2015
@@ -55,6 +55,7 @@
* #[HBase]
* #[Hive]
* #[Yarn]
+ * #[Storm]
* #[Limitations]
* #[Troubleshooting]
* #[Export Controls]
Modified: knox/trunk/books/0.6.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_service-details.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_service-details.md (original)
+++ knox/trunk/books/0.6.0/book_service-details.md Mon May 4 18:52:17 2015
@@ -35,6 +35,7 @@ These are the current Hadoop services wi
* #[HBase]
* #[Hive]
* #[Yarn]
+* #[Storm]
### Assumptions
Modified: knox/trunk/books/0.6.0/dev-guide/book.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/dev-guide/book.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/dev-guide/book.md (original)
+++ knox/trunk/books/0.6.0/dev-guide/book.md Mon May 4 18:52:17 2015
@@ -116,11 +116,11 @@ This is also true of the other -release
| gateway-provider-security-shiro | Shiro authentiation integration. |
| gateway-provider-security-webappsec | Filters to prevent common webapp security issues. |
| gateway-service-as | The implementation of the Access service POC. |
+| gateway-service-definitions | The implementation of the Service definition and rewrite files. |
| gateway-service-hbase | The implementation of the HBase service. |
| gateway-service-hive | The implementation of the Hive service. |
| gateway-service-oozie | The implementation of the Oozie service. |
| gateway-service-tgs | The implementation of the Ticket Granting service POC. |
-| gateway-service-webhcat | The implementation of the WebHCat service. |
| gateway-service-webhdfs | The implementation of the WebHdfs service. |
| gateway-server | The implementation of the Knox gateway server. |
| gateway-shell | The implementation of the Knox Groovy shell. |
@@ -805,6 +805,202 @@ allRules.addRules( newRules );
</project>
```
+#### Service Definition Files ####
+
+As of release 0.6.0, the gateway now also supports a declarative way of plugging-in a new Service. A Service can be defined with a
+combination of two files, these are:
+
+ service.xml
+ rewrite.xml
+
+The rewrite.xml file contains the rewrite rules as defined in other sections of this guide, and the service.xml file contains
+the various routes (paths) to be provided by the Service and the rewrite rule bindings to those paths. This will be described in further detail
+in this section.
+
+While the service.xml file is absolutely required, the rewrite.xml file in theory is optional (though it is highly unlikely that no rewrite rules
+are needed).
+
+To add a new service, simply add a service.xml and rewrite.xml file in an appropriate directory (see #[Service Definition Directory Structure])
+in the module gateway-service-definitions to make the new service part of the Knox build.
+
+##### service.xml #####
+
+Below is a sample of a very simple service.xml file, taking the same weather api example.
+
+```xml
+<service role="WEATHER" name="weather" version="0.1.0">
+ <routes>
+ <route path="/weather/**?**"/>
+ </routes>
+</service>
+
+```
+
+**service**
+: The root tag is 'service' that has the three required attributes: *role*, *name* and *version*. These three values disambiguate this
+service definition from others. To ensure the exact same service definition is being used in a topology file, all values should be specified,
+
+```xml
+<topology>
+ <gateway>
+ ....
+ </gateway>
+ <service>
+ <role>WEATHER</role>
+ <name>weather</name>
+ <version>0.1.0</version>
+ <url>http://api.openweathermap.org/data</url>
+ </service>
+ ....
+</topology>
+```
+
+If only *role* is specified in the topology file (the only required element other than *url*) then the first service definition of that role found
+will be used with the highest version of that role and name. Similarly if only the *version* is omitted from the topology specification of the service,
+the service definition of the highest version will be used. It is therefore important to specify a version for a service if it is desired that
+a topology be locked down to a specific version of a service.
+
+
+**routes**
+: Wrapper element for one or more routes.
+
+**route**
+: A route specifies the *path* that the service is routing as well as any rewrite bindings or policy bindings. Another child element that may be used here
+is a *dispatch* element.
+
+**rewrite**
+: A rewrite rule or function that is to be applied to the path. A rewrite element contains a *apply* attribute that references the rewrite function or rule
+by name. Along with the *apply* attribute, a *to* attribute must be used. The *to* specifies what part of the request or response to rewrite.
+The valid values for the *to* attribute are:
+
+- request.url
+- request.headers
+- request.cookies
+- request.body
+- response.headers
+- response.cookies
+- response.body
+
+Below is an example of a snippet from the WebHDFS service definition
+
+```xml
+ <route path="/webhdfs/v1/**?**">
+ <rewrite apply="WEBHDFS/webhdfs/inbound/namenode/file" to="request.url"/>
+ <rewrite apply="WEBHDFS/webhdfs/outbound/namenode/headers" to="response.headers"/>
+ </route>
+```
+
+**dispatch**
+: The dispatch element can be used to plug-in a custom dispatch class. The interface for Dispatch can be found in the module
+gateway-spi, org.apache.hadoop.gateway.dispatch.Dispatch.
+
+This element can be used at the service level (i.e. as a child of the service tag) or at the route level. A dispatch specified
+at the route level takes precedence over a dispatch specified at the service level. By default the dispatch used is
+org.apache.hadoop.gateway.dispatch.DefaultDispatch.
+
+The dispatch tag has four attributes that can be specified.
+
+*contributor-name* : This attribute can be used to specify a deployment contributor to be invoked for a custom dispatch.
+
+*classname* : This attribute can be used to specify a custom dispatch class.
+
+*ha-contributor-name* : This attribute can be used to specify a deployment contributor to be invoked for custom HA dispatch functionality.
+
+*ha-classname* : This attribute can be used to specify a custom dispatch class with HA functionality.
+
+Only one of contributor-name or classname should be specified and one of ha-contributor-name or ha-classname should be specified.
+
+If providing a custom dispatch, either a jar should be provided, see #[Class Path] or a #[Maven Module] should be created.
+
+**policies**
+: This is a wrapper tag for *policy* elements and can be a child of the *service* tag or the *route* tag. Once again, just like with
+dispatch, the route level policies defined override the ones at the service level.
+
+This element can contain one or more *policy* elements. The order of the *policy* elements is important as that will be the
+order of execution.
+
+**policy**
+: At this time the policy element just has two attributes, *role* and *name*. These are used to execute a deployment contributor
+by that role and name. Therefore new policies must be added by using the deployment contributor mechanism.
+
+For example,
+
+```xml
+<service role="FOO" name="foo" version="1.0.0">
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="authentication"/>
+ <policy role="rewrite"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ </policies>
+ <routes>
+ <route path="/foo/?**">
+ <rewrite apply="FOO/foo/inbound" to="request.url"/>
+ <policies>
+ <policy role="webappsec"/>
+ <policy role="federation"/>
+ <policy role="identity-assertion"/>
+ <policy role="authorization"/>
+ <policy role="rewrite"/>
+ </policies>
+ <dispatch contributor-name="http-client" />
+ </route>
+ </routes>
+ <dispatch contributor-name="custom-client" ha-contributor-name="ha-client"/>
+</service>
+```
+
+##### rewrite.xml #####
+
+The rewrite.xml file that accompanies the service.xml file follows the same rules as described in the section #[Rewrite Provider].
+
+#### Service Definition Directory Structure ####
+
+On installation of the Knox gateway, the following directory structure can be found under ${GATEWAY_HOME}/data. This
+is a mirror of the directories and files under the module gateway-service-definitions.
+
+ services
+ |______ service name
+ |______ version
+ |______service.xml
+ |______rewrite.xml
+
+
+For example,
+
+ services
+ |______ webhdfs
+ |______ 2.4.0
+ |______service.xml
+ |______rewrite.xml
+
+
+To test out a new service, you can just add the appropriate files (service.xml and rewrite.xml) in a directory under
+${GATEWAY_HOME}/data/services. If you want to make the service contribution to the Knox build, they files need to go
+in the gateway-service-definitions module.
+
+#### Service Definition Runtime Behavior ####
+
+The runtime artifacts as well as the behavior does not change whether the service is plugged in via the deployment
+descriptors or through a service.xml file.
+
+#### Custom Dispatch Dependency Injection ####
+
+When writing a custom dispatch class, one often needs configuration or gateway services. A lightweight dependency injection
+system is used that can inject instances of classes or primitives available in the filter configuration's init params or
+as a servlet context attribute.
+
+Details of this can be found in the module gateway-util-configinjector and also an example use of it is in the class
+org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following method for example:
+
+```java
+ @Configure
+ protected void setReplayBufferSize(@Default("8") int size) {
+ replayBufferSize = size;
+ }
+```
+
### Providers ###
```java
Modified: knox/trunk/books/0.6.0/service_storm.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/service_storm.md?rev=1677654&r1=1677653&r2=1677654&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/service_storm.md (original)
+++ knox/trunk/books/0.6.0/service_storm.md Mon May 4 18:52:17 2015
@@ -15,7 +15,7 @@
limitations under the License.
--->
-#### Storm ###
+### Storm ###
Storm is a distributed realtime computation system. Storm exposes REST APIs for UI functionality that can be used for
retrieving metrics data and configuration information as well as management operations such as starting or stopping topologies.
@@ -60,7 +60,7 @@ For the log viewer the mapping is as fol
| Cluster | `http://{storm-logviewer-host}:{storm-logviewer-port}` |
-#### Storm Examples via cURL
+#### Storm Examples
Some of the various calls that can be made and examples using curl are listed below.