You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by su...@apache.org on 2015/09/22 18:22:26 UTC
svn commit: r1704665 - in /knox: site/books/knox-0-7-0/ trunk/books/0.7.0/
Author: sumit
Date: Tue Sep 22 16:22:23 2015
New Revision: 1704665
URL: http://svn.apache.org/viewvc?rev=1704665&view=rev
Log:
KNOX-580 docs for default HA support
Added:
knox/trunk/books/0.7.0/service_default_ha.md
Modified:
knox/site/books/knox-0-7-0/user-guide.html
knox/trunk/books/0.7.0/book_service-details.md
knox/trunk/books/0.7.0/service_hbase.md
knox/trunk/books/0.7.0/service_hive.md
knox/trunk/books/0.7.0/service_oozie.md
knox/trunk/books/0.7.0/service_webhcat.md
Modified: knox/site/books/knox-0-7-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/user-guide.html?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/site/books/knox-0-7-0/user-guide.html (original)
+++ knox/site/books/knox-0-7-0/user-guide.html Tue Sep 22 16:22:23 2015
@@ -2478,7 +2478,7 @@ session.shutdown()
<ul>
<li><code>Job.queryStatus(session).jobId(jobId).now().string</code></li>
</ul></li>
-</ul><h3><a id="Oozie"></a>Oozie</h3><p>Oozie is a Hadoop component provides complex job workflows to be submitted and managed. Please refer to the latest <a href="http://oozie.apache.org/docs/4.0.0/">Oozie documentation</a> for details.</p><p>In order to make Oozie accessible via the gateway there are several important Hadoop configuration settings. These all relate to the network endpoint exposed by various Hadoop services.</p><p>The HTTP endpoint at which Oozie is running can be found via the oozie.base.url property in the oozie-site.xml file. In a Sandbox installation this can typically be found in /etc/oozie/conf/oozie-site.xml.</p>
+</ul><h3><a id="WebHCat+HA"></a>WebHCat HA</h3><p>Please look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a id="Oozie"></a>Oozie</h3><p>Oozie is a Hadoop component provides complex job workflows to be submitted and managed. Please refer to the latest <a href="http://oozie.apache.org/docs/4.0.0/">Oozie documentation</a> for details.</p><p>In order to make Oozie accessible via the gateway there are several important Hadoop configuration settings. These all relate to the network endpoint exposed by various Hadoop services.</p><p>The HTTP endpoint at which Oozie is running can be found via the oozie.base.url property in the oozie-site.xml file. In a Sandbox installation this can typically be found in /etc/oozie/conf/oozie-site.xml.</p>
<pre><code><property>
<name>oozie.base.url</name>
<value>http://sandbox.hortonworks.com:11000/oozie</value>
@@ -2603,7 +2603,7 @@ curl -i -k -u guest:guest-password -X DE
<ul>
<li><code>Workflow.status(session).jobId(jobId).now().string</code></li>
</ul></li>
-</ul><h3><a id="HBase"></a>HBase</h3><p>The HBase REST API is provided by the Stargate service for HBase. See the HBase Stargate Setup section below for getting started with stargate and Knox with the Hortonworks Sandbox environment.</p><h4><a id="HBase+URL+Mapping"></a>HBase URL Mapping</h4>
+</ul><h3><a id="Oozie+HA"></a>Oozie HA</h3><p>Please look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a id="HBase"></a>HBase</h3><p>The HBase REST API is provided by the Stargate service for HBase. See the HBase Stargate Setup section below for getting started with stargate and Knox with the Hortonworks Sandbox environment.</p><h4><a id="HBase+URL+Mapping"></a>HBase URL Mapping</h4>
<table>
<tbody>
<tr>
@@ -3177,7 +3177,7 @@ session.shutdown(10, SECONDS)
</code></pre><h4><a id="Delete+table"></a>Delete table</h4>
<pre><code>curl -ik -u guest:guest-password\
-X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table1/schema'
-</code></pre><h3><a id="Hive"></a>Hive</h3><p>The <a href="https://cwiki.apache.org/confluence/display/Hive/Home">Hive wiki pages</a> describe Hive installation and configuration processes. In sandbox configuration file for Hive is located at /etc/hive/hive-site.xml. Hive Server has to be started in HTTP mode. Note the properties shown below as they are related to configuration required by the gateway.</p>
+</code></pre><h3><a id="HBase+REST+HA"></a>HBase REST HA</h3><p>Please look at <a href="#Default+Service+HA+support">Default Service HA support</a></p><h3><a id="Hive"></a>Hive</h3><p>The <a href="https://cwiki.apache.org/confluence/display/Hive/Home">Hive wiki pages</a> describe Hive installation and configuration processes. In sandbox configuration file for Hive is located at /etc/hive/hive-site.xml. Hive Server has to be started in HTTP mode. Note the properties shown below as they are related to configuration required by the gateway.</p>
<pre><code><property>
<name>hive.server2.thrift.http.port</name>
<value>10001</value>
@@ -3410,7 +3410,7 @@ connection.close();
<li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
<li><p>zookeeperEnsemble - A comma separated list of host names (or IP addresses) of the zookeeper hosts that consist of the ensemble that the Hive servers register their information with. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.zookeeper.quorum’.</p></li>
<li><p>zookeeperNamespace - This is the namespace under which HiveServer2 information is registered in the Zookeeper ensemble. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.server2.zookeeper.namespace’.</p></li>
-</ul><p>And for the service configuration itself the additional URLs that standby nodes should be added to the list. The active URL (at the time of configuration) should ideally be added to the top of the list.</p>
+</ul><p>And for the service configuration itself the URLs need not be added to the list. For example.</p>
<pre><code><service>
<role>HIVE</role>
</service>
@@ -3582,6 +3582,56 @@ curl -ik -b ~/cookiejar.txt -c ~/cookiej
curl -ik -b ~/cookiejar.txt -c ~/cookiejar.txt -u guest:guest-password -H 'x-csrf-token:{token-value}' -X POST \
http://localhost:8744/api/v1/topology/{id}/kill/0
+</code></pre><h3><a id="Default+Service+HA+support"></a>Default Service HA support</h3><p>Knox provides connectivity based failover functionality for service calls that can be made to more than one server instance in a cluster. To enable this functionality HaProvider configuration needs to be enabled for the service and the service itself needs to be configured with more than one URL in the topology file.</p><p>The default HA functionality works on a simple round robin algorithm, where the top of the list of URLs is always used to route all of a service’s REST calls until a connection error occurs. The top URL is then put at the bottom of the list and the next URL is attempted. This goes on until the setting of ‘maxFailoverAttempts’ is reached.</p><p>At present the following services can avail this default High Availability functionality and have been tested for the same:</p>
+<ul>
+ <li>WEBHCAT</li>
+ <li>HBASE</li>
+ <li>OOZIE</li>
+</ul><p>To enable HA functionality for a service in Knox the following configuration has to be added to the topology file.</p>
+<pre><code><provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>{SERVICE}</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+</provider>
+</code></pre><p>The role and name of the provider above must be as shown. The name in the ‘param’ section i.e. {SERVICE} must match that of the service role name that is being configured for HA and the value in the ‘param’ section is the configuration for that particular service in HA mode. For example, the value of {SERVICE} can be ‘WEBHCAT’, ‘HBASE’ or ‘OOZIE’.</p><p>To configure multiple services in HA mode, additional ‘param’ sections can be added.</p><p>For example,</p>
+<pre><code><provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>OOZIE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>HBASE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>WEBHCAT</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+</provider>
+</code></pre><p>The various configuration parameters are described below:</p>
+<ul>
+ <li><p>maxFailoverAttempts - This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried again.</p></li>
+ <li><p>failoverSleep - The amount of time in millis that the process will wait or sleep before attempting to failover.</p></li>
+ <li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
+</ul><p>And for the service configuration itself the additional URLs should be added to the list.</p>
+<pre><code><service>
+ <role>{SERVICE}</role>
+ <url>http://host1:port1</url>
+ <url>http://host2:port2</url>
+</service>
+</code></pre><p>For example,</p>
+<pre><code><service>
+ <role>OOZIE</role>
+ <url>http://sandbox1:11000/oozie</url>
+ <url>http://sandbox2:11000/oozie</url>
+</service>
</code></pre><h3><a id="Service+Test+API"></a>Service Test API</h3><p>The gateway supports a Service Test API that can be used to test Knox’s ability to connect to each of the different Hadoop services via a simeple HTTP GET request. To be able to access this API one must add the following line into the topology for which you wish to run the service test.</p>
<pre><code><service>
<role>SERVICE-TEST</role>
Modified: knox/trunk/books/0.7.0/book_service-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/book_service-details.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/book_service-details.md (original)
+++ knox/trunk/books/0.7.0/book_service-details.md Tue Sep 22 16:22:23 2015
@@ -82,6 +82,7 @@ Therefore each request via cURL will res
<<service_hive.md>>
<<service_yarn.md>>
<<service_storm.md>>
+<<service_default_ha.md>>
### Service Test API
Added: knox/trunk/books/0.7.0/service_default_ha.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_default_ha.md?rev=1704665&view=auto
==============================================================================
--- knox/trunk/books/0.7.0/service_default_ha.md (added)
+++ knox/trunk/books/0.7.0/service_default_ha.md Tue Sep 22 16:22:23 2015
@@ -0,0 +1,102 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+--->
+
+### Default Service HA support ###
+
+Knox provides connectivity based failover functionality for service calls that can be made to more than one server
+instance in a cluster. To enable this functionality HaProvider configuration needs to be enabled for the service and
+the service itself needs to be configured with more than one URL in the topology file.
+
+The default HA functionality works on a simple round robin algorithm, where the top of the list of URLs is always used
+to route all of a service's REST calls until a connection error occurs. The top URL is then put at the bottom of the
+list and the next URL is attempted. This goes on until the setting of 'maxFailoverAttempts' is reached.
+
+At present the following services can avail this default High Availability functionality and have been tested for the
+same:
+
+* WEBHCAT
+* HBASE
+* OOZIE
+
+To enable HA functionality for a service in Knox the following configuration has to be added to the topology file.
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>{SERVICE}</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section i.e. {SERVICE} must match
+that of the service role name that is being configured for HA and the value in the 'param' section is the configuration
+for that particular service in HA mode. For example, the value of {SERVICE} can be 'WEBHCAT', 'HBASE' or 'OOZIE'.
+
+To configure multiple services in HA mode, additional 'param' sections can be added.
+
+For example,
+
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>OOZIE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>HBASE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ <param>
+ <name>WEBHCAT</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true</value>
+ </param>
+ </provider>
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried
+again.
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+And for the service configuration itself the additional URLs should be added to the list.
+
+ <service>
+ <role>{SERVICE}</role>
+ <url>http://host1:port1</url>
+ <url>http://host2:port2</url>
+ </service>
+
+For example,
+
+ <service>
+ <role>OOZIE</role>
+ <url>http://sandbox1:11000/oozie</url>
+ <url>http://sandbox2:11000/oozie</url>
+ </service>
Modified: knox/trunk/books/0.7.0/service_hbase.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hbase.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hbase.md (original)
+++ knox/trunk/books/0.7.0/service_hbase.md Tue Sep 22 16:22:23 2015
@@ -649,3 +649,9 @@ Scanner URL will be in Location response
curl -ik -u guest:guest-password\
-X DELETE 'https://localhost:8443/gateway/sandbox/hbase/table1/schema'
+
+
+### HBase REST HA ###
+
+Please look at #[Default Service HA support]
+
Modified: knox/trunk/books/0.7.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hive.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hive.md (original)
+++ knox/trunk/books/0.7.0/service_hive.md Tue Sep 22 16:22:23 2015
@@ -315,9 +315,7 @@ This is the namespace under which HiveSe
obtained from Hive's config file hive-site.xml as the value for the parameter 'hive.server2.zookeeper.namespace'.
-And for the service configuration itself the additional URLs that standby nodes should be added to the list. The active
-URL (at the time of configuration) should ideally be added to the top of the list.
-
+And for the service configuration itself the URLs need not be added to the list. For example.
<service>
<role>HIVE</role>
Modified: knox/trunk/books/0.7.0/service_oozie.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_oozie.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_oozie.md (original)
+++ knox/trunk/books/0.7.0/service_oozie.md Tue Sep 22 16:22:23 2015
@@ -190,4 +190,9 @@ These replacement values are identified
* Example
* `Workflow.status(session).jobId(jobId).now().string`
+### Oozie HA ###
+
+Please look at #[Default Service HA support]
+
+
Modified: knox/trunk/books/0.7.0/service_webhcat.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_webhcat.md?rev=1704665&r1=1704664&r2=1704665&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_webhcat.md (original)
+++ knox/trunk/books/0.7.0/service_webhcat.md Tue Sep 22 16:22:23 2015
@@ -150,3 +150,8 @@ Each line from the file `samples/Example
* BasicResponse
* Example
* `Job.queryStatus(session).jobId(jobId).now().string`
+
+### WebHCat HA ###
+
+Please look at #[Default Service HA support]
+