You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by mo...@apache.org on 2017/11/06 20:54:45 UTC
svn commit: r1814441 - in /knox: site/books/knox-0-14-0/user-guide.html
site/index.html site/issue-tracking.html site/license.html
site/mail-lists.html site/project-info.html site/team-list.html
trunk/books/0.14.0/config.md
Author: more
Date: Mon Nov 6 20:54:44 2017
New Revision: 1814441
URL: http://svn.apache.org/viewvc?rev=1814441&view=rev
Log:
KNOX-1103 - Document Recommendation for Using Shared Provider Configuration With HA Providers (Phil Zampino via Sandeep More)
Modified:
knox/site/books/knox-0-14-0/user-guide.html
knox/site/index.html
knox/site/issue-tracking.html
knox/site/license.html
knox/site/mail-lists.html
knox/site/project-info.html
knox/site/team-list.html
knox/trunk/books/0.14.0/config.md
Modified: knox/site/books/knox-0-14-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-14-0/user-guide.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/books/knox-0-14-0/user-guide.html (original)
+++ knox/site/books/knox-0-14-0/user-guide.html Mon Nov 6 20:54:44 2017
@@ -828,7 +828,16 @@ ip-10-39-107-209.ec2.internal
<param><name>localhost</name><value>sandbox,sandbox.hortonworks.com</value></param>
</provider>
</gateway>
-</code></pre><h5><a id="Simplified+Descriptor+Files">Simplified Descriptor Files</a> <a href="#Simplified+Descriptor+Files"><img src="markbook-section-link.png"/></a></h5><p>Simplified descriptors allow service URLs to be defined explicitly, just like full topology descriptors. However, if URLs are omitted for a service, Knox will attempt to discover that service’s URLs from the Hadoop cluster. Currently, this behavior is only supported for clusters managed by Ambari. In any case, the simplified descriptors are much more concise than a full topology descriptor.</p><p><em>Descriptor Properties</em></p>
+</code></pre><h6><a id="Sharing+HA+Providers">Sharing HA Providers</a> <a href="#Sharing+HA+Providers"><img src="markbook-section-link.png"/></a></h6><p>HA Providers are a special concern with respect to sharing provider configuration because they include service-specific (and possibly cluster-specific) configuration.</p><p>This requires extra attention because the service configurations corresponding to the associated HA Provider configuration must contain the correct content to function properly.</p><p>For a shared provider configuration with an HA Provider service:</p>
+<ul>
+ <li>If the referencing descriptor does not declare the corresponding service, then the HA Provider configuration is effectively ignored since the service isn’t exposed by the topology.</li>
+ <li>If a corresponding service is declared in the descriptor
+ <ul>
+ <li>If service endpoint discovery is employed, then Knox should populate the URLs correctly to support the HA behavior.</li>
+ <li>Otherwise, the URLs must be explicitly specified for that service in the descriptor.</li>
+ </ul></li>
+ <li>If the descriptor content is correct, but the cluster service is not configured for HA, then the HA behavior obviously won’t work.</li>
+</ul><p><em>Apache ZooKeeper-based HA Provider Services</em></p><p>The HA Provider configuration for some services (e.g., <a href="#HiveServer2+HA">HiveServer2</a>, <a href="#Kafka+HA">Kafka</a>) includes references to Apache ZooKeeper hosts (i.e., the ZooKeeper ensemble). It’s important to understand the relationship of that ensemble configuration to the topologies referencing it. These ZooKeeper ensembles are often cluster-specific. If the ZooKeeper ensemble in the provider configuration is part of cluster A, then it’s probably incorrect to reference it in a topology for cluster B since the Hadoop service endpoints will probably be the wrong ones. However, if multiple clusters are working with a common ZooKeeper ensemble, then sharing this provider configuration <em>may</em> be appropriate. Note that service endpoint discovery does <em>not</em> handle these ZooKeeper ensemble details; they are static provider configuration.</p><p>Be sure to pay extra attention when sha
ring HA Provider configuration across topologies.</p><h5><a id="Simplified+Descriptor+Files">Simplified Descriptor Files</a> <a href="#Simplified+Descriptor+Files"><img src="markbook-section-link.png"/></a></h5><p>Simplified descriptors allow service URLs to be defined explicitly, just like full topology descriptors. However, if URLs are omitted for a service, Knox will attempt to discover that service’s URLs from the Hadoop cluster. Currently, this behavior is only supported for clusters managed by Ambari. In any case, the simplified descriptors are much more concise than a full topology descriptor.</p><p><em>Descriptor Properties</em></p>
<table>
<thead>
<tr>
Modified: knox/site/index.html
URL: http://svn.apache.org/viewvc/knox/site/index.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/index.html (original)
+++ knox/site/index.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – REST API and Application Gateway for the Apache Hadoop Ecosystem</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/site/issue-tracking.html
URL: http://svn.apache.org/viewvc/knox/site/issue-tracking.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/issue-tracking.html (original)
+++ knox/site/issue-tracking.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Issue Tracking</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/site/license.html
URL: http://svn.apache.org/viewvc/knox/site/license.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/license.html (original)
+++ knox/site/license.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project License</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/site/mail-lists.html
URL: http://svn.apache.org/viewvc/knox/site/mail-lists.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/mail-lists.html (original)
+++ knox/site/mail-lists.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Mailing Lists</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/site/project-info.html
URL: http://svn.apache.org/viewvc/knox/site/project-info.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/project-info.html (original)
+++ knox/site/project-info.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Project Information</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/site/team-list.html
URL: http://svn.apache.org/viewvc/knox/site/team-list.html?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/site/team-list.html (original)
+++ knox/site/team-list.html Mon Nov 6 20:54:44 2017
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2017-11-03
+ | Generated by Apache Maven Doxia at 2017-11-06
| Rendered using Apache Maven Fluido Skin 1.3.0
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20171103" />
+ <meta name="Date-Revision-yyyymmdd" content="20171106" />
<meta http-equiv="Content-Language" content="en" />
<title>Knox Gateway – Team list</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" />
@@ -58,7 +58,7 @@
- <li id="publishDate" class="pull-right">Last Published: 2017-11-03</li>
+ <li id="publishDate" class="pull-right">Last Published: 2017-11-06</li>
</ul>
</div>
Modified: knox/trunk/books/0.14.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/config.md?rev=1814441&r1=1814440&r2=1814441&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/config.md (original)
+++ knox/trunk/books/0.14.0/config.md Mon Nov 6 20:54:44 2017
@@ -424,6 +424,32 @@ difference is that it's defined in its o
</provider>
</gateway>
+
+###### Sharing HA Providers ######
+
+HA Providers are a special concern with respect to sharing provider configuration because they include service-specific (and possibly cluster-specific) configuration.
+
+This requires extra attention because the service configurations corresponding to the associated HA Provider configuration must contain the correct content to function properly.
+
+For a shared provider configuration with an HA Provider service:
+
+- If the referencing descriptor does not declare the corresponding service, then the HA Provider configuration is effectively ignored since the service isn't exposed by the topology.
+- If a corresponding service is declared in the descriptor
+ - If service endpoint discovery is employed, then Knox should populate the URLs correctly to support the HA behavior.
+ - Otherwise, the URLs must be explicitly specified for that service in the descriptor.
+- If the descriptor content is correct, but the cluster service is not configured for HA, then the HA behavior obviously won't work.
+
+*Apache ZooKeeper-based HA Provider Services*
+
+The HA Provider configuration for some services (e.g., [HiveServer2](#HiveServer2+HA), [Kafka](#Kafka+HA)) includes references to Apache ZooKeeper hosts (i.e., the ZooKeeper ensemble).
+It's important to understand the relationship of that ensemble configuration to the topologies referencing it. These ZooKeeper ensembles are often cluster-specific. If the ZooKeeper
+ensemble in the provider configuration is part of cluster A, then it's probably incorrect to reference it in a topology for cluster B since the Hadoop service endpoints will probably
+be the wrong ones. However, if multiple clusters are working with a common ZooKeeper ensemble, then sharing this provider configuration *may* be appropriate. Note that service endpoint
+discovery does *not* handle these ZooKeeper ensemble details; they are static provider configuration.
+
+Be sure to pay extra attention when sharing HA Provider configuration across topologies.
+
+
##### Simplified Descriptor Files #####
Simplified descriptors allow service URLs to be defined explicitly, just like full topology descriptors. However, if URLs are