You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by km...@apache.org on 2015/10/26 16:54:15 UTC
svn commit: r1710635 [7/11] - in /knox: site/ site/books/knox-0-3-0/
site/books/knox-0-4-0/ site/books/knox-0-5-0/ site/books/knox-0-6-0/
site/books/knox-0-7-0/ site/images/
trunk/markbook/src/main/java/org/apache/hadoop/gateway/markbook/
trunk/markboo...
Modified: knox/site/books/knox-0-6-0/dev-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/dev-guide.html?rev=1710635&r1=1710634&r2=1710635&view=diff
==============================================================================
--- knox/site/books/knox-0-6-0/dev-guide.html (original)
+++ knox/site/books/knox-0-6-0/dev-guide.html Mon Oct 26 15:54:14 2015
@@ -13,7 +13,7 @@
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---><p><link href="book.css" rel="stylesheet"/></p><p><img src="knox-logo.gif" alt="Knox"/> <img src="apache-logo.gif" align="right" alt="Apache"/></p><h1><a id="Apache+Knox+Gateway+0.6.x+Developer's+Guide"></a>Apache Knox Gateway 0.6.x Developer’s Guide</h1><h2><a id="Table+Of+Contents"></a>Table Of Contents</h2>
+--><p><link href="book.css" rel="stylesheet"/></p><p><img src="knox-logo.gif" alt="Knox"/> <img src="apache-logo.gif" align="right" alt="Apache"/></p><h1><a id="Apache+Knox+Gateway+0.6.x+Developer's+Guide">Apache Knox Gateway 0.6.x Developer’s Guide</a> <a href="#Apache+Knox+Gateway+0.6.x+Developer's+Guide"><img src="markbook-section-link.png"/></a></h1><h2><a id="Table+Of+Contents">Table Of Contents</a> <a href="#Table+Of+Contents"><img src="markbook-section-link.png"/></a></h2>
<ul>
<li><a href="#Overview">Overview</a></li>
<li><a href="#Architecture+Overview">Architecture Overview</a></li>
@@ -30,7 +30,7 @@
<li><a href="#Auditing">Auditing</a></li>
<li><a href="#Logging">Logging</a></li>
<li><a href="#Internationalization">Internationalization</a></li>
-</ul><h2><a id="Overview"></a>Overview</h2><p>Apache Knox gateway is a specialized reverse proxy gateway for various Hadoop REST APIs. However, the gateway is built entirely upon a fairly generic framework. This framework is used to “plug-in” all of the behavior that makes it specific to Hadoop in general and any particular Hadoop REST API. It would be equally as possible to create a customized reverse proxy for other non-Hadoop HTTP endpoints. This approach is taken to ensure that the Apache Knox gateway can scale with the rapidly evolving Hadoop ecosystem.</p><p>Throughout this guide we will be using a publicly available REST API to demonstrate the development of various extension mechanisms. <a href="http://openweathermap.org/">http://openweathermap.org/</a></p><h3><a id="Architecture+Overview"></a>Architecture Overview</h3><p>The gateway itself is a layer over an embedded Jetty JEE server. At the very highest level the gateway processes requests by using request URLs
to lookup specific JEE Servlet Filter chain that is used to process the request. The gateway framework provides extensible mechanisms to assemble chains of custom filters that support secured access to services.</p><p>The gateway has two primary extensibility mechanisms: Service and Provider. The Service extensibility framework provides a way to add support for new HTTP/REST endpoints. For example, the support for WebHdfs is plugged into the Knox gateway as a Service. The Provider extensibility framework allows adding new features to the gateway that can be used across Services. An example of a Provider is an authentication provider. Providers can also expose APIs that other service and provider extensions can utilize.</p><p>Service and Provider integrations interact with the gateway framework in two distinct phases: Deployment and Runtime. The gateway framework can be thought of as a layer over the JEE Servlet framework. Specifically all runtime processing within the gateway is pe
rformed by JEE Servlet Filters. The two phases interact with this JEE Servlet Filter based model in very different ways. The first phase, Deployment, is responsible for converting fairly simple to understand configuration called topology into JEE WebArchive (WAR) based implementation details. The second phase, Runtime, is the processing of requests via a set of Filters configured in the WAR.</p><p>From an “ethos” perspective, Service and Provider extensions should attempt to incur complexity associated with configuration in the deployment phase. This should allow for very streamlined request processing that is very high performance and easily testable. The preference at runtime, in OO style, is for small classes that perform a specific function. The ideal set of implementation classes are then assembled by the Service and Provider plugins during deployment.</p><p>A second critical design consideration is streaming. The processing infrastructure is build around JEE Servle
t Filters as they provide a natural streaming interception model. All Provider implementations should make every attempt to maintaining this streaming characteristic.</p><h3><a id="Project+Overview"></a>Project Overview</h3><p>The table below describes the purpose of the current modules in the project. Of particular importance are the root pom.xml and the gateway-release module. The root pom.xml is critical because this is where all dependency version must be declared. There should be no dependency version information in module pom.xml files. The gateway-release module is critical because the dependencies declared there essentially define the classpath of the released gateway server. This is also true of the other -release modules in the project.</p>
+</ul><h2><a id="Overview">Overview</a> <a href="#Overview"><img src="markbook-section-link.png"/></a></h2><p>Apache Knox gateway is a specialized reverse proxy gateway for various Hadoop REST APIs. However, the gateway is built entirely upon a fairly generic framework. This framework is used to “plug-in” all of the behavior that makes it specific to Hadoop in general and any particular Hadoop REST API. It would be equally as possible to create a customized reverse proxy for other non-Hadoop HTTP endpoints. This approach is taken to ensure that the Apache Knox gateway can scale with the rapidly evolving Hadoop ecosystem.</p><p>Throughout this guide we will be using a publicly available REST API to demonstrate the development of various extension mechanisms. <a href="http://openweathermap.org/">http://openweathermap.org/</a></p><h3><a id="Architecture+Overview">Architecture Overview</a> <a href="#Architecture+Overview"><img src="markbook-section-link.png"/></a></h3><p>The
gateway itself is a layer over an embedded Jetty JEE server. At the very highest level the gateway processes requests by using request URLs to lookup specific JEE Servlet Filter chain that is used to process the request. The gateway framework provides extensible mechanisms to assemble chains of custom filters that support secured access to services.</p><p>The gateway has two primary extensibility mechanisms: Service and Provider. The Service extensibility framework provides a way to add support for new HTTP/REST endpoints. For example, the support for WebHdfs is plugged into the Knox gateway as a Service. The Provider extensibility framework allows adding new features to the gateway that can be used across Services. An example of a Provider is an authentication provider. Providers can also expose APIs that other service and provider extensions can utilize.</p><p>Service and Provider integrations interact with the gateway framework in two distinct phases: Deployment and Runtime. The
gateway framework can be thought of as a layer over the JEE Servlet framework. Specifically all runtime processing within the gateway is performed by JEE Servlet Filters. The two phases interact with this JEE Servlet Filter based model in very different ways. The first phase, Deployment, is responsible for converting fairly simple to understand configuration called topology into JEE WebArchive (WAR) based implementation details. The second phase, Runtime, is the processing of requests via a set of Filters configured in the WAR.</p><p>From an “ethos” perspective, Service and Provider extensions should attempt to incur complexity associated with configuration in the deployment phase. This should allow for very streamlined request processing that is very high performance and easily testable. The preference at runtime, in OO style, is for small classes that perform a specific function. The ideal set of implementation classes are then assembled by the Service and Provider plu
gins during deployment.</p><p>A second critical design consideration is streaming. The processing infrastructure is build around JEE Servlet Filters as they provide a natural streaming interception model. All Provider implementations should make every attempt to maintaining this streaming characteristic.</p><h3><a id="Project+Overview">Project Overview</a> <a href="#Project+Overview"><img src="markbook-section-link.png"/></a></h3><p>The table below describes the purpose of the current modules in the project. Of particular importance are the root pom.xml and the gateway-release module. The root pom.xml is critical because this is where all dependency version must be declared. There should be no dependency version information in module pom.xml files. The gateway-release module is critical because the dependencies declared there essentially define the classpath of the released gateway server. This is also true of the other -release modules in the project.</p>
<table>
<thead>
<tr>
@@ -216,7 +216,7 @@
<td>A collection of utility for building and releasing. </td>
</tr>
</tbody>
-</table><h3><a id="Development+Processes"></a>Development Processes</h3><p>The project uses Maven in general with a few convenience Ant targets.</p><p>Building the project can be built via Maven or Ant. The two commands below are equivalent.</p>
+</table><h3><a id="Development+Processes">Development Processes</a> <a href="#Development+Processes"><img src="markbook-section-link.png"/></a></h3><p>The project uses Maven in general with a few convenience Ant targets.</p><p>Building the project can be built via Maven or Ant. The two commands below are equivalent.</p>
<pre><code>mvn clean install
ant
</code></pre><p>A more complete build can be done that builds and generates the unsigned ZIP release artifacts. You will find these in the target/{version} directory (e.g. target/0.7.0-SNAPSHOT).</p>
@@ -228,7 +228,7 @@ ant release
<pre><code>ant start-test-servers
</code></pre><p>So putting things together the following Ant command will build a release, install it and start the servers ready for manual testing.</p>
<pre><code>ant release install-test-home start-test-servers
-</code></pre><h2><a id="Behavior"></a>Behavior</h2><p>There are two distinct phases in the behavior of the gateway. These are the deployment and runtime phases. The deployment phase is responsible for converting topology descriptors into an executable JEE style WAR. The runtime phase is the processing of requests via WAR created during the deployment phase.</p><p>The deployment phase is arguably the more complex of the two phases. This is because runtime relies on well known JEE constructs while deployment introduces new framework concepts. The base concept of the deployment framework is that of a “contributor”. In the framework, contributors are pluggable component responsible for generating JEE WAR artifacts from topology files.</p><h3><a id="Deployment+Behavior"></a>Deployment Behavior</h3><p>The goal of the deployment phase is to take easy to understand topology descriptions and convert them into optimized runtime artifacts. Our goal is not only should the topology d
escriptors be easy to understand, but have them be easy for a management system (e.g. Ambari) to generate. Think of deployment as compiling an assembly descriptor into a JEE WAR. WARs are then deployed to an embedded JEE container (i.e. Jetty).</p><p>Consider the results of starting the gateway the first time. There are two sets of files that are relevant for deployment. The first is the topology file <code><GATEWAY_HOME>/conf/topologies/sandbox.xml</code>. This second set is the WAR structure created during the deployment of the topology file.</p>
+</code></pre><h2><a id="Behavior">Behavior</a> <a href="#Behavior"><img src="markbook-section-link.png"/></a></h2><p>There are two distinct phases in the behavior of the gateway. These are the deployment and runtime phases. The deployment phase is responsible for converting topology descriptors into an executable JEE style WAR. The runtime phase is the processing of requests via WAR created during the deployment phase.</p><p>The deployment phase is arguably the more complex of the two phases. This is because runtime relies on well known JEE constructs while deployment introduces new framework concepts. The base concept of the deployment framework is that of a “contributor”. In the framework, contributors are pluggable component responsible for generating JEE WAR artifacts from topology files.</p><h3><a id="Deployment+Behavior">Deployment Behavior</a> <a href="#Deployment+Behavior"><img src="markbook-section-link.png"/></a></h3><p>The goal of the deployment phase is to ta
ke easy to understand topology descriptions and convert them into optimized runtime artifacts. Our goal is not only should the topology descriptors be easy to understand, but have them be easy for a management system (e.g. Ambari) to generate. Think of deployment as compiling an assembly descriptor into a JEE WAR. WARs are then deployed to an embedded JEE container (i.e. Jetty).</p><p>Consider the results of starting the gateway the first time. There are two sets of files that are relevant for deployment. The first is the topology file <code><GATEWAY_HOME>/conf/topologies/sandbox.xml</code>. This second set is the WAR structure created during the deployment of the topology file.</p>
<pre><code>data/deployments/sandbox.war.143bfef07f0/WEB-INF
web.xml
gateway.xml
@@ -286,7 +286,7 @@ ant release
<li><p>During this invocation the provider deployment contributor populate populate service specific information. In particular it will add filters to the gateway servlet’s runtime descriptor by adding JEE Servlet Filters. These filters will be added to the resources (or URLs) identified by the service deployment contributor.</p></li>
<li><p>The finalizeContribute method of all referenced and default provider deployment contributors is invoked.</p></li>
<li><p>The provider deployment contributor is expected to perform any final modifications to the runtime descriptors in the WAR structure.</p></li>
-</ol><h3><a id="Runtime+Behavior"></a>Runtime Behavior</h3><p>The runtime behavior of the gateway is somewhat simpler as it more or less follows well known JEE models. There is one significant wrinkle. The filter chains are managed within the GatewayServlet as opposed to being managed by the JEE container. This is the result of an early decision made in the project. The intention is to allow more powerful URL matching than is provided by the JEE Servlet mapping mechanisms.</p><p>The diagram below provides a high level overview of the runtime processing. An explanation for each step is provided after the diagram.</p><p><img src='runtime-overview.png'/></p>
+</ol><h3><a id="Runtime+Behavior">Runtime Behavior</a> <a href="#Runtime+Behavior"><img src="markbook-section-link.png"/></a></h3><p>The runtime behavior of the gateway is somewhat simpler as it more or less follows well known JEE models. There is one significant wrinkle. The filter chains are managed within the GatewayServlet as opposed to being managed by the JEE container. This is the result of an early decision made in the project. The intention is to allow more powerful URL matching than is provided by the JEE Servlet mapping mechanisms.</p><p>The diagram below provides a high level overview of the runtime processing. An explanation for each step is provided after the diagram.</p><p><img src='runtime-overview.png'/></p>
<ol>
<li><p>A REST client makes a HTTP request that is received by the embedded JEE container.</p></li>
<li><p>A filter chain is looked up in a map of URLs to filter chains.</p></li>
@@ -306,7 +306,7 @@ ant release
<li><p>The GatewayFilter invokes the doFilter method on the selected chain.</p></li>
<li><p>The chain invokes the doFilter method of the first filter in the chain.</p></li>
<li><p>Each filter in the chain continues processing by invoking the doFilter on the next filter in the chain. Ultimately a dispatch filter forward the request to the real service instead of invoking another filter. This is sometimes referred to as pivoting.</p></li>
-</ol><h2><a id="Gateway+Servlet+&+Gateway+Filter"></a>Gateway Servlet & Gateway Filter</h2><p>TODO</p>
+</ol><h2><a id="Gateway+Servlet+&+Gateway+Filter">Gateway Servlet & Gateway Filter</a> <a href="#Gateway+Servlet+&+Gateway+Filter"><img src="markbook-section-link.png"/></a></h2><p>TODO</p>
<pre><code class="xml"><web-app>
<servlet>
@@ -365,7 +365,7 @@ public void testDevGuideSample() throws
assertThat( match.getValue(), is( "fake-chain") );
}
-</code></pre><h2><a id="Extension+Logistics"></a>Extension Logistics</h2><p>There are a number of extension points available in the gateway: services, providers, rewrite steps and functions, etc. All of these use the Java ServiceLoader mechanism for their discovery. There are two ways to make these extensions available on the class path at runtime. The first way to to add a new module to the project and have the extension “built-in”. The second is to add the extension to the class path of the server after it is installed. Both mechanism are described in more detail below.</p><h3><a id="Service+Loaders"></a>Service Loaders</h3><p>Extensions are discovered via Java’s [Service Loader|http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html] mechanism. There are good [tutorials|http://docs.oracle.com/javase/tutorial/ext/basics/spi.html] available for learning more about this. The basics come town to two things.</p>
+</code></pre><h2><a id="Extension+Logistics">Extension Logistics</a> <a href="#Extension+Logistics"><img src="markbook-section-link.png"/></a></h2><p>There are a number of extension points available in the gateway: services, providers, rewrite steps and functions, etc. All of these use the Java ServiceLoader mechanism for their discovery. There are two ways to make these extensions available on the class path at runtime. The first way to to add a new module to the project and have the extension “built-in”. The second is to add the extension to the class path of the server after it is installed. Both mechanism are described in more detail below.</p><h3><a id="Service+Loaders">Service Loaders</a> <a href="#Service+Loaders"><img src="markbook-section-link.png"/></a></h3><p>Extensions are discovered via Java’s [Service Loader|http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html] mechanism. There are good [tutorials|http://docs.oracle.com/javase/tutoria
l/ext/basics/spi.html] available for learning more about this. The basics come town to two things.</p>
<ol>
<li><p>Implement the service contract interface (e.g. ServiceDeploymentContributor, ProviderDeploymentContributor)</p></li>
<li><p>Create a file in META-INF/services of the JAR that will contain the extension. This file will be named as the fully qualified name of the contract interface (e.g. org.apache.hadoop.gateway.deploy.ProviderDeploymentContributor). The contents of the file will be the fully qualified names of any implementation of that contract interface in that JAR.</p></li>
@@ -383,17 +383,17 @@ public void testDevGuideSample() throws
}
fail( "Failed to find " + ShiroDeploymentContributor.class.getName() + " via service loader." );
}
-</code></pre><h3><a id="Class+Path"></a>Class Path</h3><p>One way to extend the functionality of the server without having to recompile is to add the extension JARs to the servers class path. As an extensible server this is made straight forward but it requires some understanding of how the server’s classpath is setup. In the <GATEWAY_HOME> directory there are four class path related directories (i.e. bin, lib, dep, ext).</p><p>The bin directory contains very small “launcher” jars that contain only enough code to read configuration and setup a class path. By default the configuration of a launcher is embedded with the launcher JAR but it may also be extracted into a .cfg file. In that file you will see how the class path is defined.</p>
+</code></pre><h3><a id="Class+Path">Class Path</a> <a href="#Class+Path"><img src="markbook-section-link.png"/></a></h3><p>One way to extend the functionality of the server without having to recompile is to add the extension JARs to the servers class path. As an extensible server this is made straight forward but it requires some understanding of how the server’s classpath is setup. In the <GATEWAY_HOME> directory there are four class path related directories (i.e. bin, lib, dep, ext).</p><p>The bin directory contains very small “launcher” jars that contain only enough code to read configuration and setup a class path. By default the configuration of a launcher is embedded with the launcher JAR but it may also be extracted into a .cfg file. In that file you will see how the class path is defined.</p>
<pre><code>class.path=../lib/*.jar,../dep/*.jar;../ext;../ext/*.jar
</code></pre><p>The paths are all relative to the directory that contains the launcher JAR.</p>
<dl><dt>../lib/*.jar</dt><dd>These are the “built-in” jars that are part of the project itself. Information is provided elsewhere in this document for how to integrate a built-in extension.</dd><dt>../dep/*.jar</dt><dd>These are the JARs for all of the external dependencies of the project. This separation between the generated JARs and dependencies help keep licensing issues straight.</dd><dt>../ext</dt><dd>This directory is for post-install extensions and is empty by default. Including the directory (vs *.jar) allows for individual classes to be placed in this directory.
</dd><dt>../ext/*.jar</dt><dd>This would pick up all extension JARs placed in the ext directory.</dd>
-</dl><p>Note that order is significant. The lib JARs take precedence over dep JARs and they take precedence over ext classes and JARs.</p><h3><a id="Maven+Module"></a>Maven Module</h3><p>Integrating an extension into the project follows well established Maven patterns for adding modules. Below are several points that are somewhat unique to the Knox project.</p>
+</dl><p>Note that order is significant. The lib JARs take precedence over dep JARs and they take precedence over ext classes and JARs.</p><h3><a id="Maven+Module">Maven Module</a> <a href="#Maven+Module"><img src="markbook-section-link.png"/></a></h3><p>Integrating an extension into the project follows well established Maven patterns for adding modules. Below are several points that are somewhat unique to the Knox project.</p>
<ol>
<li><p>Add the module to the root pom.xml file’s <modules> list. Take care to ensure that the module is in the correct place in the list based on its dependencies. Note: In general modules should not have non-test dependencies on gateway-server but rather gateway-spi</p></li>
<li><p>Any new dependencies must be represented in the root pom.xml file’s <dependencyManagement> section. The required version of the dependencies will be declared there. The new sub-module’s pom.xml file must not include dependency version information. This helps prevent dependency version conflict issues.</p></li>
<li><p>If the extension is to be “built into” the released gateway server it needs to be added as a dependency to the gateway-release module. This is done by adding to the <dependencies> section of the gateway-release’s pom.xml file. If this isn’t done the JARs for the module will not be automatically packaged into the release artifacts. This can be useful while an extension is under development but not yet ready for inclusion in the release.</p></li>
-</ol><p>More detailed examples of adding both a service and a provider extension are provided in subsequent sections.</p><h3><a id="Services"></a>Services</h3><p>Services are extensions that are responsible for converting information in the topology file to runtime descriptors. Typically services do not require their own runtime descriptors. Rather, they modify either the gateway runtime descriptor (i.e. gateway.xml) or descriptors of other providers (e.g. rewrite.xml).</p><p>The service provider interface for a Service is ServiceDeploymentContributor and is shown below.</p>
+</ol><p>More detailed examples of adding both a service and a provider extension are provided in subsequent sections.</p><h3><a id="Services">Services</a> <a href="#Services"><img src="markbook-section-link.png"/></a></h3><p>Services are extensions that are responsible for converting information in the topology file to runtime descriptors. Typically services do not require their own runtime descriptors. Rather, they modify either the gateway runtime descriptor (i.e. gateway.xml) or descriptors of other providers (e.g. rewrite.xml).</p><p>The service provider interface for a Service is ServiceDeploymentContributor and is shown below.</p>
<pre><code class="java">package org.apache.hadoop.gateway.deploy;
import org.apache.hadoop.gateway.topology.Service;
public interface ServiceDeploymentContributor {
@@ -426,7 +426,7 @@ public interface ServiceDeploymentContri
}
</code></pre>
<dl><dt>void initializeContribution( DeploymentContext context );</dt><dd>In this method a contributor would create, initialize and add any descriptors it was responsible for to the deployment context. For the weather service example this isn’t required so the empty method isn’t shown here.</dd><dt>void contributeService( DeploymentContext context, Service service ) throws Exception;</dt><dd>In this method a service contributor typically add and configures any features it requires. This method will be dissected in more detail below.</dd><dt>void finalizeContribution( DeploymentContext context );</dt><dd>In this method a contributor would finalize any descriptors it was responsible for to the deployment context. For the weather service example this isn’t required so the empty method isn’t shown here.</dd>
-</dl><h4><a id="Service+Contribution+Behavior"></a>Service Contribution Behavior</h4><p>In order to understand the job of the ServiceDeploymentContributor a few runtime descriptors need to be introduced.</p>
+</dl><h4><a id="Service+Contribution+Behavior">Service Contribution Behavior</a> <a href="#Service+Contribution+Behavior"><img src="markbook-section-link.png"/></a></h4><p>In order to understand the job of the ServiceDeploymentContributor a few runtime descriptors need to be introduced.</p>
<dl><dt>Gateway Runtime Descriptor: WEB-INF/gateway.xml</dt><dd>This runtime descriptor controls the behavior of the GatewayFilter. It defines a mapping between resources (i.e. URL patterns) and filter chains. The sample gateway runtime descriptor helps illustrate.</dd>
</dl>
<pre><code class="xml"><gateway>
@@ -521,10 +521,10 @@ public interface ServiceDeploymentContri
</dependencies>
</project>
-</code></pre><h4><a id="Service+Definition+Files"></a>Service Definition Files</h4><p>As of release 0.6.0, the gateway now also supports a declarative way of plugging-in a new Service. A Service can be defined with a combination of two files, these are:</p>
+</code></pre><h4><a id="Service+Definition+Files">Service Definition Files</a> <a href="#Service+Definition+Files"><img src="markbook-section-link.png"/></a></h4><p>As of release 0.6.0, the gateway now also supports a declarative way of plugging-in a new Service. A Service can be defined with a combination of two files, these are:</p>
<pre><code>service.xml
rewrite.xml
-</code></pre><p>The rewrite.xml file contains the rewrite rules as defined in other sections of this guide, and the service.xml file contains the various routes (paths) to be provided by the Service and the rewrite rule bindings to those paths. This will be described in further detail in this section.</p><p>While the service.xml file is absolutely required, the rewrite.xml file in theory is optional (though it is highly unlikely that no rewrite rules are needed).</p><p>To add a new service, simply add a service.xml and rewrite.xml file in an appropriate directory (see <a href="#Service+Definition+Directory+Structure">Service Definition Directory Structure</a>) in the module gateway-service-definitions to make the new service part of the Knox build.</p><h5><a id="service.xml"></a>service.xml</h5><p>Below is a sample of a very simple service.xml file, taking the same weather api example.</p>
+</code></pre><p>The rewrite.xml file contains the rewrite rules as defined in other sections of this guide, and the service.xml file contains the various routes (paths) to be provided by the Service and the rewrite rule bindings to those paths. This will be described in further detail in this section.</p><p>While the service.xml file is absolutely required, the rewrite.xml file in theory is optional (though it is highly unlikely that no rewrite rules are needed).</p><p>To add a new service, simply add a service.xml and rewrite.xml file in an appropriate directory (see <a href="#Service+Definition+Directory+Structure">Service Definition Directory Structure</a>) in the module gateway-service-definitions to make the new service part of the Knox build.</p><h5><a id="service.xml">service.xml</a> <a href="#service.xml"><img src="markbook-section-link.png"/></a></h5><p>Below is a sample of a very simple service.xml file, taking the same weather api example.</p>
<pre><code class="xml"><service role="WEATHER" name="weather" version="0.1.0">
<routes>
<route path="/weather/**?**"/>
@@ -592,7 +592,7 @@ rewrite.xml
</routes>
<dispatch contributor-name="custom-client" ha-contributor-name="ha-client"/>
</service>
-</code></pre><h5><a id="rewrite.xml"></a>rewrite.xml</h5><p>The rewrite.xml file that accompanies the service.xml file follows the same rules as described in the section <a href="#Rewrite+Provider">Rewrite Provider</a>.</p><h4><a id="Service+Definition+Directory+Structure"></a>Service Definition Directory Structure</h4><p>On installation of the Knox gateway, the following directory structure can be found under ${GATEWAY_HOME}/data. This is a mirror of the directories and files under the module gateway-service-definitions.</p>
+</code></pre><h5><a id="rewrite.xml">rewrite.xml</a> <a href="#rewrite.xml"><img src="markbook-section-link.png"/></a></h5><p>The rewrite.xml file that accompanies the service.xml file follows the same rules as described in the section <a href="#Rewrite+Provider">Rewrite Provider</a>.</p><h4><a id="Service+Definition+Directory+Structure">Service Definition Directory Structure</a> <a href="#Service+Definition+Directory+Structure"><img src="markbook-section-link.png"/></a></h4><p>On installation of the Knox gateway, the following directory structure can be found under ${GATEWAY_HOME}/data. This is a mirror of the directories and files under the module gateway-service-definitions.</p>
<pre><code>services
|______ service name
|______ version
@@ -604,12 +604,12 @@ rewrite.xml
|______ 2.4.0
|______service.xml
|______rewrite.xml
-</code></pre><p>To test out a new service, you can just add the appropriate files (service.xml and rewrite.xml) in a directory under ${GATEWAY_HOME}/data/services. If you want to make the service contribution to the Knox build, they files need to go in the gateway-service-definitions module.</p><h4><a id="Service+Definition+Runtime+Behavior"></a>Service Definition Runtime Behavior</h4><p>The runtime artifacts as well as the behavior does not change whether the service is plugged in via the deployment descriptors or through a service.xml file.</p><h4><a id="Custom+Dispatch+Dependency+Injection"></a>Custom Dispatch Dependency Injection</h4><p>When writing a custom dispatch class, one often needs configuration or gateway services. A lightweight dependency injection system is used that can inject instances of classes or primitives available in the filter configuration’s init params or as a servlet context attribute.</p><p>Details of this can be found in the module gateway-util-con
figinjector and also an example use of it is in the class org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following method for example:</p>
+</code></pre><p>To test out a new service, you can just add the appropriate files (service.xml and rewrite.xml) in a directory under ${GATEWAY_HOME}/data/services. If you want to make the service contribution to the Knox build, they files need to go in the gateway-service-definitions module.</p><h4><a id="Service+Definition+Runtime+Behavior">Service Definition Runtime Behavior</a> <a href="#Service+Definition+Runtime+Behavior"><img src="markbook-section-link.png"/></a></h4><p>The runtime artifacts as well as the behavior does not change whether the service is plugged in via the deployment descriptors or through a service.xml file.</p><h4><a id="Custom+Dispatch+Dependency+Injection">Custom Dispatch Dependency Injection</a> <a href="#Custom+Dispatch+Dependency+Injection"><img src="markbook-section-link.png"/></a></h4><p>When writing a custom dispatch class, one often needs configuration or gateway services. A lightweight dependency injection system is used that can inject instances of
classes or primitives available in the filter configuration’s init params or as a servlet context attribute.</p><p>Details of this can be found in the module gateway-util-configinjector and also an example use of it is in the class org.apache.hadoop.gateway.dispatch.DefaultDispatch. Look at the following method for example:</p>
<pre><code class="java"> @Configure
protected void setReplayBufferSize(@Default("8") int size) {
replayBufferSize = size;
}
-</code></pre><h3><a id="Providers"></a>Providers</h3>
+</code></pre><h3><a id="Providers">Providers</a> <a href="#Providers"><img src="markbook-section-link.png"/></a></h3>
<pre><code class="java">public interface ProviderDeploymentContributor {
String getRole();
String getName();
@@ -654,7 +654,7 @@ rewrite.xml
</dependencies>
</project>
-</code></pre><h3><a id="Deployment+Context"></a>Deployment Context</h3>
+</code></pre><h3><a id="Deployment+Context">Deployment Context</a> <a href="#Deployment+Context"><img src="markbook-section-link.png"/></a></h3>
<pre><code class="java">package org.apache.hadoop.gateway.deploy;
import ...
@@ -716,7 +716,7 @@ public interface DeploymentContext {
ResourceDescriptor createResource();
void addResource( ResourceDescriptor resource );
}
-</code></pre><h3><a id="Gateway+Services"></a>Gateway Services</h3><p>TODO - Describe the service registry and other global services.</p><h2><a id="Standard+Providers"></a>Standard Providers</h2><h3><a id="Rewrite+Provider"></a>Rewrite Provider</h3><p>gateway-provider-rewrite org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteRulesDescriptor</p>
+</code></pre><h3><a id="Gateway+Services">Gateway Services</a> <a href="#Gateway+Services"><img src="markbook-section-link.png"/></a></h3><p>TODO - Describe the service registry and other global services.</p><h2><a id="Standard+Providers">Standard Providers</a> <a href="#Standard+Providers"><img src="markbook-section-link.png"/></a></h2><h3><a id="Rewrite+Provider">Rewrite Provider</a> <a href="#Rewrite+Provider"><img src="markbook-section-link.png"/></a></h3><p>gateway-provider-rewrite org.apache.hadoop.gateway.filter.rewrite.api.UrlRewriteRulesDescriptor</p>
<pre><code class="xml"><rules>
<rule
dir="IN"
@@ -792,7 +792,7 @@ public void testDevGuideSampleWithEvalua
outputUri.toString(),
is( "http://api.openweathermap.org/data/2.5/weather?q=Palo+Alto" ) );
}
-</code></pre><h4><a id="Rewrite+Filters"></a>Rewrite Filters</h4><p>TODO - Cover the supported content types. TODO - Provide a XML and JSON “properties” example where one NVP is modified based on value of another name.</p>
+</code></pre><h4><a id="Rewrite+Filters">Rewrite Filters</a> <a href="#Rewrite+Filters"><img src="markbook-section-link.png"/></a></h4><p>TODO - Cover the supported content types. TODO - Provide a XML and JSON “properties” example where one NVP is modified based on value of another name.</p>
<pre><code class="xml"><rules>
<filter name="WEBHBASE/webhbase/regions/outbound">
<content type="*/json">
@@ -827,14 +827,14 @@ public void testDevGuideSampleWithEvalua
params = new ArrayList<FilterParamDescriptor>();
params.add( regionResource.createFilterParam().name( "response.body" ).value( "WEBHBASE/webhbase/regions/outbound" ) );
addRewriteFilter( context, service, regionResource, params );
-</code></p><h4><a id="Rewrite+Functions"></a>Rewrite Functions</h4><p>TODO - Provide an lowercase function as an example.</p>
+</code></p><h4><a id="Rewrite+Functions">Rewrite Functions</a> <a href="#Rewrite+Functions"><img src="markbook-section-link.png"/></a></h4><p>TODO - Provide an lowercase function as an example.</p>
<pre><code class="xml"><rules>
<functions>
<hostmap config="/WEB-INF/hostmap.txt"/>
</functions>
...
</rules>
-</code></pre><h4><a id="Rewrite+Steps"></a>Rewrite Steps</h4><p>TODO - Provide an lowercase step as an example.</p>
+</code></pre><h4><a id="Rewrite+Steps">Rewrite Steps</a> <a href="#Rewrite+Steps"><img src="markbook-section-link.png"/></a></h4><p>TODO - Provide an lowercase step as an example.</p>
<pre><code class="xml"><rules>
<rule dir="OUT" name="WEBHDFS/webhdfs/outbound/namenode/headers/location">
<match pattern="{scheme}://{host}:{port}/{path=**}?{**}"/>
@@ -842,7 +842,7 @@ public void testDevGuideSampleWithEvalua
<encrypt-query/>
</rule>
</rules>
-</code></pre><h3><a id="Identity+Assertion+Provider"></a>Identity Assertion Provider</h3><p>Adding a new identity assertion provider is as simple as extending the AbstractIdentityAsserterDeploymentContributor and the CommonIdentityAssertionFilter from the gateway-provider-identity-assertion-common module to initialize any specific configuration from filter init params and implement two methods:</p>
+</code></pre><h3><a id="Identity+Assertion+Provider">Identity Assertion Provider</a> <a href="#Identity+Assertion+Provider"><img src="markbook-section-link.png"/></a></h3><p>Adding a new identity assertion provider is as simple as extending the AbstractIdentityAsserterDeploymentContributor and the CommonIdentityAssertionFilter from the gateway-provider-identity-assertion-common module to initialize any specific configuration from filter init params and implement two methods:</p>
<ol>
<li>String mapUserPrincipal(String principalName);</li>
<li>String[] mapGroupPrincipals(String principalName, Subject subject);</li>
@@ -902,7 +902,7 @@ public class CaseShifterIdentityAssertio
<li>looks for specific filter init parameters for configuration of whether to convert to upper or to lower case</li>
<li>it no-ops the mapGroupPrincipals so that it returns null. This indicates that there are no changes needed to the groups contained within the Subject. If there are groups then they should be continued to flow through the system unchanged. This is actually the same implementation as the base class and is therefore not required to be overridden. We include it here for illustration.</li>
<li>based upon the configuration interrogated in the init method the principalName is convert to either upper or lower case.</li>
-</ol><p>That is the extent of what is needed to implement a new identity assertion provider module.</p><h3><a id="Jersey+Provider"></a>Jersey Provider</h3><p>TODO</p><h2><a id="Auditing"></a>Auditing</h2>
+</ol><p>That is the extent of what is needed to implement a new identity assertion provider module.</p><h3><a id="Jersey+Provider">Jersey Provider</a> <a href="#Jersey+Provider"><img src="markbook-section-link.png"/></a></h3><p>TODO</p><h2><a id="Auditing">Auditing</a> <a href="#Auditing"><img src="markbook-section-link.png"/></a></h2>
<pre><code class="java">public class AuditingSample {
private static Auditor AUDITOR = AuditServiceFactory.getAuditService().getAuditor(
@@ -915,7 +915,7 @@ public class CaseShifterIdentityAssertio
}
}
-</code></pre><h2><a id="Logging"></a>Logging</h2>
+</code></pre><h2><a id="Logging">Logging</a> <a href="#Logging"><img src="markbook-section-link.png"/></a></h2>
<pre><code class="java">@Messages( logger = "org.apache.project.module" )
public interface CustomMessages {
@@ -935,7 +935,7 @@ public interface CustomMessages {
}
}
-</code></pre><h2><a id="Internationalization"></a>Internationalization</h2>
+</code></pre><h2><a id="Internationalization">Internationalization</a> <a href="#Internationalization"><img src="markbook-section-link.png"/></a></h2>
<pre><code class="java">@Resources
public interface CustomResources {
@@ -955,7 +955,7 @@ public interface CustomResources {
}
}
-</code></pre><h2><a id="Trademarks"></a>Trademarks</h2><p>Apache Knox, Apache Knox Gateway, Apache, the Apache feather logo and the Apache Knox Gateway project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p><h2><a id="License"></a>License</h2><p>Apache Knox uses the standard <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache license</a>.</p><h2><a id="Privacy+Policy"></a>Privacy Policy</h2><p>Apache Knox uses the standard Apache privacy policy.</p><p>Information about your use of this website is collected using server access logs and a tracking cookie. The collected information consists of the following:</p>
+</code></pre><h2><a id="Trademarks">Trademarks</a> <a href="#Trademarks"><img src="markbook-section-link.png"/></a></h2><p>Apache Knox, Apache Knox Gateway, Apache, the Apache feather logo and the Apache Knox Gateway project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p><h2><a id="License">License</a> <a href="#License"><img src="markbook-section-link.png"/></a></h2><p>Apache Knox uses the standard <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache license</a>.</p><h2><a id="Privacy+Policy">Privacy Policy</a> <a href="#Privacy+Policy"><img src="markbook-section-link.png"/></a></h2><p>Apache Knox uses the standard Apache privacy policy.</p><p>Information about your use of this website is collected using server access logs and a tracking cookie. The collected information consists of the following:</p>
<ul>
<li>The IP address from which you access the website;</li>
<li>The type of browser and operating system you use to access our site;</li>
Added: knox/site/books/knox-0-6-0/markbook-section-link.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/markbook-section-link.png?rev=1710635&view=auto
==============================================================================
Binary file - no diff available.
Propchange: knox/site/books/knox-0-6-0/markbook-section-link.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: knox/site/books/knox-0-6-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-overview.png?rev=1710635&r1=1710634&r2=1710635&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-request-processing.png?rev=1710635&r1=1710634&r2=1710635&view=diff
==============================================================================
Binary files - no diff available.