You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by mo...@apache.org on 2017/12/01 21:12:53 UTC

svn commit: r1816909 - in /knox: site/books/knox-0-14-0/user-guide.html trunk/books/0.14.0/config.md

Author: more
Date: Fri Dec  1 21:12:53 2017
New Revision: 1816909

URL: http://svn.apache.org/viewvc?rev=1816909&view=rev
Log:
KNOX-1123 - Document Remote Registry Client Service and ZooKeeper Config Monitor ( Phil Zampino  via Sandeep More)

Modified:
    knox/site/books/knox-0-14-0/user-guide.html
    knox/trunk/books/0.14.0/config.md

Modified: knox/site/books/knox-0-14-0/user-guide.html
URL: http://svn.apache.org/viewvc/knox/site/books/knox-0-14-0/user-guide.html?rev=1816909&r1=1816908&r2=1816909&view=diff
==============================================================================
--- knox/site/books/knox-0-14-0/user-guide.html (original)
+++ knox/site/books/knox-0-14-0/user-guide.html Fri Dec  1 21:12:53 2017
@@ -699,6 +699,16 @@ https://{gateway-host}:{gateway-port}/{g
       <td>Excludes a comma separated list of protocols to not accept for SSL or &ldquo;none&rdquo;</td>
       <td>SSLv3</td>
     </tr>
+    <tr>
+      <td>gateway.remote.config.monitor.client</td>
+      <td>A reference to the <a href="#Remote+Configuration+Registry+Clients">remote configuration registry client</a> the remote configuration monitor will employ.</td>
+      <td>null</td>
+    </tr>
+    <tr>
+      <td>gateway.remote.config.registry.<b>&lt;name&gt;</b></td>
+      <td>A named <a href="#Remote+Configuration+Registry+Clients">remote configuration registry client</a> definition</td>
+      <td>null</td>
+    </tr>
   </tbody>
 </table><h4><a id="Topology+Descriptors">Topology Descriptors</a> <a href="#Topology+Descriptors"><img src="markbook-section-link.png"/></a></h4><p>The topology descriptor files provide the gateway with per-cluster configuration information. This includes configuration for both the providers within the gateway and the services within the Hadoop cluster. These files are located in <code>{GATEWAY_HOME}/conf/topologies</code>. The general outline of this document looks like this.</p>
 <pre><code>&lt;topology&gt;
@@ -961,7 +971,96 @@ services:
     {&quot;name&quot;:&quot;AMBARIUI&quot;, &quot;urls&quot;:[&quot;http://sandbox.hortonworks.com:8080&quot;]}
   ]
 }
-</code></pre><p>Both of these examples illustrate the specification of credentials for the interaction with Ambari. If no credentials are specified, then the default aliases are queried. Use of the default aliases is sufficient for scenarios where topology discovery will only interact with a single Ambari instance. For multiple Ambari instances however, it&rsquo;s most likely that each will require different sets of credentials. The discovery-user and discovery-pwd-alias properties exist for this purpose. Note that whether using the default credential aliases or specifying a custom password alias, these <a href="#Alias+creation">aliases must be defined</a> prior to any attempt to deploy a topology using a simplified descriptor.</p><h5><a id="Deployment+Directories">Deployment Directories</a> <a href="#Deployment+Directories"><img src="markbook-section-link.png"/></a></h5><p>Effecting topology changes is as simple as modifying files in two specific directories.</p><p>The <code>{GATEW
 AY_HOME}/conf/shared-providers/</code> directory is the location where Knox looks for provider configurations. This directory is monitored for changes, such that modifying a provider configuration file therein will trigger updates to any referencing simplified descriptors in the <code>{GATEWAY_HOME}/conf/descriptors/</code> directory. <em>Care should be taken when deleting these files if there are referencing descriptors; any subsequent modifications of referencing descriptors will fail when the deleted provider configuration cannot be found. The references should all be modified before deleting the provider configuration.</em></p><p>Likewise, the <code>{GATEWAY_HOME}/conf/descriptors/</code> directory is monitored for changes, such that adding or modifying a simplified descriptor file in this directory will trigger the generation and deployment of a topology descriptor. Deleting a descriptor from this directory will conversely result in the removal of the previously-generated topol
 ogy descriptor, and the associated topology will be undeployed.</p><p>If the service details for a deployed (generated) topology are changed in the cluster, then the Knox topology can be updated by &rsquo;touch&rsquo;ing the simplified descriptor. This will trigger discovery and regeneration/redeployment of the topology descriptor.</p><p>Note that deleting a generated topology descriptor from <code>{GATEWAY_HOME}/conf/topologies/</code> is not sufficient for its removal. If the source descriptor is modified, or Knox is restarted, the topology descriptor will be regenerated and deployed. Removing generated topology descriptors should be done by removing the associated simplified descriptor. For the same reason, editing generated topology descriptors is strongly discouraged since they can be inadvertently overwritten.</p><p>Another means by which these topology changes can be effected is the <a href="#Admin+API">Admin API</a>.</p><h4><a id="Logging">Logging</a> <a href="#Logging"><img
  src="markbook-section-link.png"/></a></h4><p>If necessary you can enable additional logging by editing the <code>log4j.properties</code> file in the <code>conf</code> directory. Changing the <code>rootLogger</code> value from <code>ERROR</code> to <code>DEBUG</code> will generate a large amount of debug logging. A number of useful, more fine loggers are also provided in the file.</p><h4><a id="Java+VM+Options">Java VM Options</a> <a href="#Java+VM+Options"><img src="markbook-section-link.png"/></a></h4><p>TODO - Java VM options doc.</p><h4><a id="Persisting+the+Master+Secret">Persisting the Master Secret</a> <a href="#Persisting+the+Master+Secret"><img src="markbook-section-link.png"/></a></h4><p>The master secret is required to start the server. This secret is used to access secured artifacts by the gateway instance. Keystore, trust stores and credential stores are all protected with the master secret.</p><p>You may persist the master secret by supplying the <em>-persist-master</e
 m> switch at startup. This will result in a warning indicating that persisting the secret is less secure than providing it at startup. We do make some provisions in order to protect the persisted password.</p><p>It is encrypted with AES 128 bit encryption and where possible the file permissions are set to only be accessible by the user that the gateway is running as.</p><p>After persisting the secret, ensure that the file at data/security/master has the appropriate permissions set for your environment. This is probably the most important layer of defense for master secret. Do not assume that the encryption is sufficient protection.</p><p>A specific user should be created to run the gateway. This user will be the only user with permissions for the persisted master file.</p><p>See the Knox CLI section for descriptions of the command line utilities related to the master secret.</p><h4><a id="Management+of+Security+Artifacts">Management of Security Artifacts</a> <a href="#Management+of+
 Security+Artifacts"><img src="markbook-section-link.png"/></a></h4><p>There are a number of artifacts that are used by the gateway in ensuring the security of wire level communications, access to protected resources and the encryption of sensitive data. These artifacts can be managed from outside of the gateway instances or generated and populated by the gateway instance itself.</p><p>The following is a description of how this is coordinated with both standalone (development, demo, etc) gateway instances and instances as part of a cluster of gateways in mind.</p><p>Upon start of the gateway server we:</p>
+</code></pre><p>Both of these examples illustrate the specification of credentials for the interaction with Ambari. If no credentials are specified, then the default aliases are queried. Use of the default aliases is sufficient for scenarios where topology discovery will only interact with a single Ambari instance. For multiple Ambari instances however, it&rsquo;s most likely that each will require different sets of credentials. The discovery-user and discovery-pwd-alias properties exist for this purpose. Note that whether using the default credential aliases or specifying a custom password alias, these <a href="#Alias+creation">aliases must be defined</a> prior to any attempt to deploy a topology using a simplified descriptor.</p><h5><a id="Deployment+Directories">Deployment Directories</a> <a href="#Deployment+Directories"><img src="markbook-section-link.png"/></a></h5><p>Effecting topology changes is as simple as modifying files in two specific directories.</p><p>The <code>{GATEW
 AY_HOME}/conf/shared-providers/</code> directory is the location where Knox looks for provider configurations. This directory is monitored for changes, such that modifying a provider configuration file therein will trigger updates to any referencing simplified descriptors in the <code>{GATEWAY_HOME}/conf/descriptors/</code> directory. <em>Care should be taken when deleting these files if there are referencing descriptors; any subsequent modifications of referencing descriptors will fail when the deleted provider configuration cannot be found. The references should all be modified before deleting the provider configuration.</em></p><p>Likewise, the <code>{GATEWAY_HOME}/conf/descriptors/</code> directory is monitored for changes, such that adding or modifying a simplified descriptor file in this directory will trigger the generation and deployment of a topology descriptor. Deleting a descriptor from this directory will conversely result in the removal of the previously-generated topol
 ogy descriptor, and the associated topology will be undeployed.</p><p>If the service details for a deployed (generated) topology are changed in the cluster, then the Knox topology can be updated by &rsquo;touch&rsquo;ing the simplified descriptor. This will trigger discovery and regeneration/redeployment of the topology descriptor.</p><p>Note that deleting a generated topology descriptor from <code>{GATEWAY_HOME}/conf/topologies/</code> is not sufficient for its removal. If the source descriptor is modified, or Knox is restarted, the topology descriptor will be regenerated and deployed. Removing generated topology descriptors should be done by removing the associated simplified descriptor. For the same reason, editing generated topology descriptors is strongly discouraged since they can be inadvertently overwritten.</p><p>Another means by which these topology changes can be effected is the <a href="#Admin+API">Admin API</a>.</p><h5><a id="Remote+Configuration+Monitor">Remote Configu
 ration Monitor</a> <a href="#Remote+Configuration+Monitor"><img src="markbook-section-link.png"/></a></h5><p>In addition to monitoring local directories for provider configurations and simplified descriptors, the gateway similarly supports monitoring ZooKeeper.</p><p>This monitor depends on a <a href="#Remote+Configuration+Registry+Clients">remote configuration registry client</a>, and that client must be specified by setting the following property in gateway-site.xml</p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.monitor.client&lt;/name&gt;
+    &lt;value&gt;sandbox-zookeeper-client&lt;/value&gt;
+    &lt;description&gt;Remote configuration monitor client name.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p>This client identifier is a reference to a remote configuration registry client, as in this example (also defined in gateway-site.xml)</p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.registry.sandbox-zookeeper-client&lt;/name&gt;
+    &lt;value&gt;type=ZooKeeper;address=localhost:2181&lt;/value&gt;
+    &lt;description&gt;ZooKeeper configuration registry client details.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p><em>The actual name of the client (e.g., sandbox-zookeeper-client) is not important, except that the reference matches the name specified in the client definition.</em></p><p>With this configuration, the gateway will monitor the following znodes in the specified ZooKeeper instance</p>
+<pre><code>/knox
+   /config
+      /shared-providers
+      /descriptors
+</code></pre><p>The creation of these znodes, and the population of their respective contents, is an activity <strong>not</strong> currently managed by the gateway. However, the <a href="#Knox+CLI">KNOX CLI</a> includes commands for managing the contents of these znodes.</p><p>These znodes are treated similarly to the local <em>shared-providers</em> and <em>descriptors</em> directories described in <a href="#Deployment+Directories">Deployment Directories</a>. When the monitor notices a change to these znodes, it will attempt to effect the same change locally.</p><p>If a provider configuration is added to the <em>/knox/config/shared-providers</em> znode, the monitor will download the new configuration to the local shared-providers directory. Likewise, if a descriptor is added to the <em>/knox/config/descriptors</em> znode, the monitor will download the new descriptor to the local descriptors directory, which will trigger an attempt to generate and deploy a corresponding topology.</p>
 <p>Modifications to the contents of these znodes, will yield the same behavior as can be seen resulting from the corresponding local modification.</p>
+<table>
+  <thead>
+    <tr>
+      <th>znode </th>
+      <th>action </th>
+      <th>result</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>/knox/config/shared-providers </td>
+      <td>add </td>
+      <td>Download the new file to the local shared-providers directory</td>
+    </tr>
+    <tr>
+      <td>/knox/config/shared-providers </td>
+      <td>modify </td>
+      <td>Download the new file to the local shared-providers directory; If there are any existing descriptor references, then topology will be regenerated and redeployed for those referencing descriptors.</td>
+    </tr>
+    <tr>
+      <td>/knox/config/shared-providers </td>
+      <td>delete </td>
+      <td>Delete the corresponding file from the local shared-providers directory</td>
+    </tr>
+    <tr>
+      <td>/knox/config/descriptors </td>
+      <td>add </td>
+      <td>Download the new file to the local descriptors directory; A corresponding topology will be generated and deployed.</td>
+    </tr>
+    <tr>
+      <td>/knox/config/descriptors </td>
+      <td>modify </td>
+      <td>Download the new file to the local descriptors directory; The corresponding topology will be regenerated and redeployed.</td>
+    </tr>
+    <tr>
+      <td>/knox/config/descriptors </td>
+      <td>delete </td>
+      <td>Delete the corresponding file from the local descriptors directory</td>
+    </tr>
+  </tbody>
+</table><p>This simplifies the configuration for HA gateway deployments, in that the gateway instances can all be configured to monitor the same ZooKeeper instance, and changes to the znodes&rsquo; contents will be applied to all those gateway instances. With this approach, it is no longer necessary to manually deploy topologies to each of the gateway instances.</p><p><em>A Note About ACLs</em></p>
+<pre><code>While the gateway does not currently require secure interactions with remote registries, it is recommended
+that ACLs be applied to restrict at least writing of the entries referenced by this monitor. If write
+access is available to everyone, then the contents of the configuration cannot be known to be trustworthy,
+and there is the potential for malicious activity. Be sure to carefully consider who will have the ability
+to define configuration in monitored remote registries, and apply the necessary measures to ensure its
+trustworthiness.
+</code></pre><h4><a id="Remote+Configuration+Registry+Clients">Remote Configuration Registry Clients</a> <a href="#Remote+Configuration+Registry+Clients"><img src="markbook-section-link.png"/></a></h4><p>One or more features of the gateway employ remote configuration registry (e.g., ZooKeeper) clients. These clients are configured by setting properties in the gateway configuration (gateway-site.xml).</p><p>Each client configuration is a single property, the name of which is prefixed with <strong>gateway.remote.config.registry.</strong> and suffixed by the client identifier. The value of such a property, is a registry-type-specific set of semicolon-delimited properties for that client, including the type of registry with which it will interact.</p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.registry.a-zookeeper-client&lt;/name&gt;
+    &lt;value&gt;type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181&lt;/value&gt;
+    &lt;description&gt;ZooKeeper configuration registry client details.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p>In the preceeding example, the client identifier is <strong>a-zookeeper-client</strong>, by way of the property name <strong>gateway.remote.config.registry.a-zookeeper-client</strong>.</p><p>The property value specifies that the client is intended to interact with ZooKeeper. It also specifies the particular ZooKeeper ensemble with which it will interact; this could be a single ZooKeeper instance as well.</p><p>The property value may also include an optional namespace, to which the client will be restricted (i.e., &ldquo;chroot&rdquo; the client).</p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.registry.a-zookeeper-client&lt;/name&gt;
+    &lt;value&gt;type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;namespace=/knox/config&lt;/value&gt;
+    &lt;description&gt;ZooKeeper configuration registry client details.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p>At least for the ZooKeeper type, authentication details may also be specified as part of the property value, for interacting with instances for which authentication is required.</p><p><strong>Digest Authentication Example</strong></p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.registry.a-zookeeper-client&lt;/name&gt;
+    &lt;value&gt;type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;authType=Digest;principal=myzkuser;credentialAlias=myzkpass&lt;/value&gt;
+    &lt;description&gt;ZooKeeper configuration registry client details.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p><strong>Kerberos Authentication Example</strong></p>
+<pre><code>&lt;property&gt;
+    &lt;name&gt;gateway.remote.config.registry.a-zookeeper-client&lt;/name&gt;
+    &lt;value&gt;type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;authType=Kerberos;principal=myzkuser;keytab=/home/user/myzk.keytab;useKeyTab=true;useTicketCache=false&lt;/value&gt;
+    &lt;description&gt;ZooKeeper configuration registry client details.&lt;/description&gt;
+&lt;/property&gt;
+</code></pre><p><em>While multiple such clients can be configured, for ZooKeeper clients, there is currently a limitation with respect to authentication. Multiple clients cannot each have distinct authentication configurations. This limitation is imposed by the underlying ZooKeeper client. Therefore, the clients must all be insecure (no authentication configured), or they must all authenticate to the same ZooKeeper using the same credentials.</em></p><p>The <a href="#Remote+Configuration+Monitor">remote configuration monitor</a> facility uses these client configurations to perform its function.</p><h4><a id="Logging">Logging</a> <a href="#Logging"><img src="markbook-section-link.png"/></a></h4><p>If necessary you can enable additional logging by editing the <code>log4j.properties</code> file in the <code>conf</code> directory. Changing the <code>rootLogger</code> value from <code>ERROR</code> to <code>DEBUG</code> will generate a large amount of debug logging. A number of useful, mo
 re fine loggers are also provided in the file.</p><h4><a id="Java+VM+Options">Java VM Options</a> <a href="#Java+VM+Options"><img src="markbook-section-link.png"/></a></h4><p>TODO - Java VM options doc.</p><h4><a id="Persisting+the+Master+Secret">Persisting the Master Secret</a> <a href="#Persisting+the+Master+Secret"><img src="markbook-section-link.png"/></a></h4><p>The master secret is required to start the server. This secret is used to access secured artifacts by the gateway instance. Keystore, trust stores and credential stores are all protected with the master secret.</p><p>You may persist the master secret by supplying the <em>-persist-master</em> switch at startup. This will result in a warning indicating that persisting the secret is less secure than providing it at startup. We do make some provisions in order to protect the persisted password.</p><p>It is encrypted with AES 128 bit encryption and where possible the file permissions are set to only be accessible by the user
  that the gateway is running as.</p><p>After persisting the secret, ensure that the file at data/security/master has the appropriate permissions set for your environment. This is probably the most important layer of defense for master secret. Do not assume that the encryption is sufficient protection.</p><p>A specific user should be created to run the gateway. This user will be the only user with permissions for the persisted master file.</p><p>See the Knox CLI section for descriptions of the command line utilities related to the master secret.</p><h4><a id="Management+of+Security+Artifacts">Management of Security Artifacts</a> <a href="#Management+of+Security+Artifacts"><img src="markbook-section-link.png"/></a></h4><p>There are a number of artifacts that are used by the gateway in ensuring the security of wire level communications, access to protected resources and the encryption of sensitive data. These artifacts can be managed from outside of the gateway instances or generated a
 nd populated by the gateway instance itself.</p><p>The following is a description of how this is coordinated with both standalone (development, demo, etc) gateway instances and instances as part of a cluster of gateways in mind.</p><p>Upon start of the gateway server we:</p>
 <ol>
   <li>Look for an identity store at <code>data/security/keystores/gateway.jks</code>.  The identity store contains the certificate and private key used to represent the identity of the server for SSL connections and signature creation.
   <ul>

Modified: knox/trunk/books/0.14.0/config.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.14.0/config.md?rev=1816909&r1=1816908&r2=1816909&view=diff
==============================================================================
--- knox/trunk/books/0.14.0/config.md (original)
+++ knox/trunk/books/0.14.0/config.md Fri Dec  1 21:12:53 2017
@@ -142,6 +142,8 @@ ssl.enabled|Indicates whether SSL is ena
 ssl.include.ciphers|A comma separated list of ciphers to accept for SSL. See the [JSSE Provider docs](http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider) for possible ciphers. These can also contain regular expressions as shown in the [Jetty documentation](http://www.eclipse.org/jetty/documentation/current/configuring-ssl.html).|all|
 ssl.exclude.ciphers|A comma separated list of ciphers to reject for SSL. See the [JSSE Provider docs](http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#SunJSSEProvider) for possible ciphers. These can also contain regular expressions as shown in the [Jetty documentation](http://www.eclipse.org/jetty/documentation/current/configuring-ssl.html).|none|
 ssl.exclude.protocols|Excludes a comma separated list of protocols to not accept for SSL or "none"|SSLv3
+gateway.remote.config.monitor.client|A reference to the [remote configuration registry client](#Remote+Configuration+Registry+Clients) the remote configuration monitor will employ.|null
+gateway.remote.config.registry.<b>&lt;name&gt;</b>|A named [remote configuration registry client](#Remote+Configuration+Registry+Clients) definition|null
 
 
 #### Topology Descriptors ####
@@ -579,6 +581,118 @@ For the same reason, editing generated t
 Another means by which these topology changes can be effected is the [Admin API](#Admin+API).
 
 
+##### Remote Configuration Monitor #####
+
+In addition to monitoring local directories for provider configurations and simplified descriptors, the gateway similarly supports monitoring ZooKeeper.
+
+This monitor depends on a [remote configuration registry client](#Remote+Configuration+Registry+Clients), and that client must be specified by setting the following property in gateway-site.xml
+
+    <property>
+        <name>gateway.remote.config.monitor.client</name>
+        <value>sandbox-zookeeper-client</value>
+        <description>Remote configuration monitor client name.</description>
+    </property>
+
+This client identifier is a reference to a remote configuration registry client, as in this example (also defined in gateway-site.xml)
+
+    <property>
+        <name>gateway.remote.config.registry.sandbox-zookeeper-client</name>
+        <value>type=ZooKeeper;address=localhost:2181</value>
+        <description>ZooKeeper configuration registry client details.</description>
+    </property>
+
+_The actual name of the client (e.g., sandbox-zookeeper-client) is not important, except that the reference matches the name specified in the client definition._
+
+With this configuration, the gateway will monitor the following znodes in the specified ZooKeeper instance
+
+    /knox
+	   /config
+	      /shared-providers
+		  /descriptors
+
+The creation of these znodes, and the population of their respective contents, is an activity __not__ currently managed by the gateway. However, the [KNOX CLI](#Knox+CLI) includes commands for managing the contents
+of these znodes.
+
+These znodes are treated similarly to the local *shared-providers* and *descriptors* directories described in [Deployment Directories](#Deployment+Directories).
+When the monitor notices a change to these znodes, it will attempt to effect the same change locally.
+
+If a provider configuration is added to the */knox/config/shared-providers* znode, the monitor will download the new configuration to the local shared-providers directory.
+Likewise, if a descriptor is added to the */knox/config/descriptors* znode, the monitor will download the new descriptor to the local descriptors directory, which will
+trigger an attempt to generate and deploy a corresponding topology.
+
+Modifications to the contents of these znodes, will yield the same behavior as can be seen resulting from the corresponding local modification.
+
+znode | action | result
+------|--------|--------
+/knox/config/shared-providers | add | Download the new file to the local shared-providers directory
+/knox/config/shared-providers | modify | Download the new file to the local shared-providers directory; If there are any existing descriptor references, then topology will be regenerated and redeployed for those referencing descriptors.
+/knox/config/shared-providers | delete | Delete the corresponding file from the local shared-providers directory
+/knox/config/descriptors | add | Download the new file to the local descriptors directory; A corresponding topology will be generated and deployed.
+/knox/config/descriptors | modify | Download the new file to the local descriptors directory; The corresponding topology will be regenerated and redeployed.
+/knox/config/descriptors | delete | Delete the corresponding file from the local descriptors directory
+
+This simplifies the configuration for HA gateway deployments, in that the gateway instances can all be configured to monitor the same ZooKeeper instance, and changes to the znodes' contents will be applied to all those gateway instances. With this approach, it is no longer necessary to manually deploy topologies to each of the gateway instances.
+
+_A Note About ACLs_
+
+    While the gateway does not currently require secure interactions with remote registries, it is recommended
+	that ACLs be applied to restrict at least writing of the entries referenced by this monitor. If write
+	access is available to everyone, then the contents of the configuration cannot be known to be trustworthy,
+	and there is the potential for malicious activity. Be sure to carefully consider who will have the ability
+	to define configuration in monitored remote registries, and apply the necessary measures to ensure its
+	trustworthiness.
+
+
+#### Remote Configuration Registry Clients ####
+
+One or more features of the gateway employ remote configuration registry (e.g., ZooKeeper) clients. These clients are configured by setting properties in the gateway configuration (gateway-site.xml).
+
+Each client configuration is a single property, the name of which is prefixed with __gateway.remote.config.registry.__ and suffixed by the client identifier.
+The value of such a property, is a registry-type-specific set of semicolon-delimited properties for that client, including the type of registry with which it will interact.
+
+    <property>
+        <name>gateway.remote.config.registry.a-zookeeper-client</name>
+        <value>type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181</value>
+        <description>ZooKeeper configuration registry client details.</description>
+    </property>
+
+In the preceeding example, the client identifier is __a-zookeeper-client__, by way of the property name __gateway.remote.config.registry.a-zookeeper-client__.
+
+The property value specifies that the client is intended to interact with ZooKeeper. It also specifies the particular ZooKeeper ensemble with which it will interact; this could be a single ZooKeeper instance as well.
+
+The property value may also include an optional namespace, to which the client will be restricted (i.e., "chroot" the client).
+
+    <property>
+        <name>gateway.remote.config.registry.a-zookeeper-client</name>
+        <value>type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;namespace=/knox/config</value>
+        <description>ZooKeeper configuration registry client details.</description>
+    </property>
+
+At least for the ZooKeeper type, authentication details may also be specified as part of the property value, for interacting with instances for which authentication is required.
+
+__Digest Authentication Example__
+
+    <property>
+        <name>gateway.remote.config.registry.a-zookeeper-client</name>
+        <value>type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;authType=Digest;principal=myzkuser;credentialAlias=myzkpass</value>
+        <description>ZooKeeper configuration registry client details.</description>
+    </property>
+
+
+__Kerberos Authentication Example__
+
+    <property>
+        <name>gateway.remote.config.registry.a-zookeeper-client</name>
+        <value>type=ZooKeeper;address=zkhost1:2181,zkhost2:2181,zkhost3:2181;authType=Kerberos;principal=myzkuser;keytab=/home/user/myzk.keytab;useKeyTab=true;useTicketCache=false</value>
+        <description>ZooKeeper configuration registry client details.</description>
+    </property>
+
+
+_While multiple such clients can be configured, for ZooKeeper clients, there is currently a limitation with respect to authentication. Multiple clients cannot each have distinct authentication configurations. This limitation is imposed by the underlying ZooKeeper client. Therefore, the clients must all be insecure (no authentication configured), or they must all authenticate to the same ZooKeeper using the same credentials._
+
+The [remote configuration monitor](#Remote+Configuration+Monitor) facility uses these client configurations to perform its function.
+
+
 #### Logging ####
 
 If necessary you can enable additional logging by editing the `log4j.properties` file in the `conf` directory.