You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by su...@apache.org on 2015/09/15 17:58:36 UTC
svn commit: r1703239 - in /knox: site/books/knox-0-4-0/
site/books/knox-0-5-0/ site/books/knox-0-6-0/ site/books/knox-0-7-0/
trunk/books/0.7.0/
Author: sumit
Date: Tue Sep 15 15:58:36 2015
New Revision: 1703239
URL: http://svn.apache.org/r1703239
Log:
KNOX-570 docs HS2 HA support
Modified:
knox/site/books/knox-0-4-0/deployment-overview.png
knox/site/books/knox-0-4-0/deployment-provider.png
knox/site/books/knox-0-4-0/deployment-service.png
knox/site/books/knox-0-4-0/runtime-overview.png
knox/site/books/knox-0-4-0/runtime-request-processing.png
knox/site/books/knox-0-5-0/deployment-overview.png
knox/site/books/knox-0-5-0/deployment-provider.png
knox/site/books/knox-0-5-0/deployment-service.png
knox/site/books/knox-0-5-0/runtime-overview.png
knox/site/books/knox-0-5-0/runtime-request-processing.png
knox/site/books/knox-0-6-0/deployment-overview.png
knox/site/books/knox-0-6-0/deployment-provider.png
knox/site/books/knox-0-6-0/deployment-service.png
knox/site/books/knox-0-6-0/runtime-overview.png
knox/site/books/knox-0-6-0/runtime-request-processing.png
knox/site/books/knox-0-7-0/deployment-overview.png
knox/site/books/knox-0-7-0/deployment-provider.png
knox/site/books/knox-0-7-0/deployment-service.png
knox/site/books/knox-0-7-0/runtime-overview.png
knox/site/books/knox-0-7-0/runtime-request-processing.png
knox/site/books/knox-0-7-0/user-guide.html
knox/trunk/books/0.7.0/service_hive.md
Modified: knox/site/books/knox-0-4-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-4-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-4-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-5-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-5-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-6-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-6-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/deployment-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/deployment-provider.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-provider.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/deployment-service.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/deployment-service.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/runtime-overview.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/runtime-overview.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/runtime-request-processing.png
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/runtime-request-processing.png?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
Binary files - no diff available.
Modified: knox/site/books/knox-0-7-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-7-0/user-guide.html?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
--- knox/site/books/knox-0-7-0/user-guide.html (original)
+++ knox/site/books/knox-0-7-0/user-guide.html Tue Sep 15 15:58:36 2015
@@ -3392,7 +3392,29 @@ connection.close();
2012-02-03 --- 18:35:34 --- SampleClass6 --- [TRACE]
2012-02-03 --- 18:35:34 --- SampleClass2 --- [DEBUG]
...
-</code></pre><h3><a id="Yarn"></a>Yarn</h3><p>Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST API’s allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.</p><p>The docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To enable this functionality, a topology file needs to have the following configuration:</p>
+</code></pre><h3><a id="HIVE+SERVER2+HA"></a>HIVE SERVER2 HA</h3><p>Knox provides basic failover functionality for calls made to Hive Server when more than one HiveServer2 instance is installed in the cluster and registered with the same Zookeeper ensemble. The HA functionality in this case fetches the HiveServer2 URL information from a Zookeeper ensemble, so the user need only supply the necessary Zookeeper configuration and not the Hive connection URLs.</p><p>To enable HA functionality for Hive in Knox the following configuration has to be added to the topology file.</p>
+<pre><code><provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>HIVE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181;
+ zookeeperNamespace=hiveserver2</value>
+ </param>
+</provider>
+</code></pre><p>The role and name of the provider above must be as shown. The name in the ‘param’ section must match that of the service role name that is being configured for HA and the value in the ‘param’ section is the configuration for that particular service in HA mode. In this case the name is ‘HIVE’.</p><p>The various configuration parameters are described below:</p>
+<ul>
+ <li><p>maxFailoverAttempts - This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried again after the list is fetched again from Zookeeper (a refresh of the list is done at this point)</p></li>
+ <li><p>failoverSleep - The amount of time in millis that the process will wait or sleep before attempting to failover.</p></li>
+ <li><p>enabled - Flag to turn the particular service on or off for HA.</p></li>
+ <li><p>zookeeperEnsemble - A comma separated list of host names (or IP addresses) of the zookeeper hosts that consist of the ensemble that the Hive servers register their information with. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.zookeeper.quorum’.</p></li>
+ <li><p>zookeeperNamespace - This is the namespace under which HiveServer2 information is registered in the Zookeeper ensemble. This value can be obtained from Hive’s config file hive-site.xml as the value for the parameter ‘hive.server2.zookeeper.namespace’.</p></li>
+</ul><p>And for the service configuration itself the additional URLs that standby nodes should be added to the list. The active URL (at the time of configuration) should ideally be added to the top of the list.</p>
+<pre><code><service>
+ <role>HIVE</role>
+</service>
+</code></pre><p>Please note that there is no <url> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.</p><h3><a id="Yarn"></a>Yarn</h3><p>Knox provides gateway functionality for the REST APIs of the ResourceManager. The ResourceManager REST API’s allow the user to get information about the cluster - status on the cluster, metrics on the cluster, scheduler information, information about nodes in the cluster, and information about applications on the cluster. Also as of hadoop version 2.5.0, the user can submit a new application as well as kill it (or get state) using the ‘Writable’ APIs.</p><p>The docs for this can be found here</p><p><a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html</a></p><p>To enable this functionality, a topology file needs to have the following configuration:</p>
<pre><code><service>
<role>RESOURCEMANAGER</role>
<url>http://<hostname>:<port>/ws</url>
Modified: knox/trunk/books/0.7.0/service_hive.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.7.0/service_hive.md?rev=1703239&r1=1703238&r2=1703239&view=diff
==============================================================================
--- knox/trunk/books/0.7.0/service_hive.md (original)
+++ knox/trunk/books/0.7.0/service_hive.md Tue Sep 15 15:58:36 2015
@@ -266,3 +266,62 @@ Expected output:
2012-02-03 --- 18:35:34 --- SampleClass6 --- [TRACE]
2012-02-03 --- 18:35:34 --- SampleClass2 --- [DEBUG]
...
+
+### HIVE SERVER2 HA ###
+
+Knox provides basic failover functionality for calls made to Hive Server when more than one HiveServer2 instance is
+installed in the cluster and registered with the same Zookeeper ensemble. The HA functionality in this case fetches the
+HiveServer2 URL information from a Zookeeper ensemble, so the user need only supply the necessary Zookeeper
+configuration and not the Hive connection URLs.
+
+To enable HA functionality for Hive in Knox the following configuration has to be added to the topology file.
+
+ <provider>
+ <role>ha</role>
+ <name>HaProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>HIVE</name>
+ <value>maxFailoverAttempts=3;failoverSleep=1000;enabled=true;zookeeperEnsemble=machine1:2181,machine2:2181,machine3:2181;
+ zookeeperNamespace=hiveserver2</value>
+ </param>
+ </provider>
+
+The role and name of the provider above must be as shown. The name in the 'param' section must match that of the service
+role name that is being configured for HA and the value in the 'param' section is the configuration for that particular
+service in HA mode. In this case the name is 'HIVE'.
+
+The various configuration parameters are described below:
+
+* maxFailoverAttempts -
+This is the maximum number of times a failover will be attempted. The failover strategy at this time is very simplistic
+in that the next URL in the list of URLs provided for the service is used and the one that failed is put at the bottom
+of the list. If the list is exhausted and the maximum number of attempts is not reached then the first URL will be tried
+again after the list is fetched again from Zookeeper (a refresh of the list is done at this point)
+
+* failoverSleep -
+The amount of time in millis that the process will wait or sleep before attempting to failover.
+
+* enabled -
+Flag to turn the particular service on or off for HA.
+
+* zookeeperEnsemble -
+A comma separated list of host names (or IP addresses) of the zookeeper hosts that consist of the ensemble that the Hive
+servers register their information with. This value can be obtained from Hive's config file hive-site.xml as the value
+for the parameter 'hive.zookeeper.quorum'.
+
+* zookeeperNamespace -
+This is the namespace under which HiveServer2 information is registered in the Zookeeper ensemble. This value can be
+obtained from Hive's config file hive-site.xml as the value for the parameter 'hive.server2.zookeeper.namespace'.
+
+
+And for the service configuration itself the additional URLs that standby nodes should be added to the list. The active
+URL (at the time of configuration) should ideally be added to the top of the list.
+
+
+ <service>
+ <role>HIVE</role>
+ </service>
+
+Please note that there is no <url> tag specified here as the URLs for the Hive servers are obtained from Zookeeper.
+