You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by ma...@apache.org on 2009/06/23 21:03:45 UTC
svn commit: r787789 - in /hadoop/zookeeper/trunk: CHANGES.txt
docs/zookeeperProgrammers.html docs/zookeeperProgrammers.pdf
src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
Author: mahadev
Date: Tue Jun 23 19:03:45 2009
New Revision: 787789
URL: http://svn.apache.org/viewvc?rev=787789&view=rev
Log:
ZOOKEEPER-329. document how to integrate 3rd party authentication into ZK server ACLs. (breed via mahadev)
Modified:
hadoop/zookeeper/trunk/CHANGES.txt
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
Modified: hadoop/zookeeper/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/CHANGES.txt?rev=787789&r1=787788&r2=787789&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/CHANGES.txt (original)
+++ hadoop/zookeeper/trunk/CHANGES.txt Tue Jun 23 19:03:45 2009
@@ -229,6 +229,9 @@
ZOOKEEPER-315. add forrest docs for bookkeeper. (flavio via mahadev)
+ ZOOKEEPER-329. document how to integrate 3rd party authentication into ZK
+server ACLs. (breed via mahadev)
+
NEW FEATURES:
ZOOKEEPER-371. jdiff documentation included in build/release (giri via phunt)
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html?rev=787789&r1=787788&r2=787789&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html Tue Jun 23 19:03:45 2009
@@ -265,6 +265,9 @@
</ul>
</li>
<li>
+<a href="#sc_ZooKeeperPluggableAuthentication">Pluggable ZooKeeper authentication</a>
+</li>
+<li>
<a href="#ch_zkGuarantees">Consistency Guarantees</a>
</li>
<li>
@@ -1012,12 +1015,18 @@
(other). ZooKeeper does not have a notion of an owner of a
znode. Instead, an ACL specifies sets of ids and permissions that
are associated with those ids.</p>
+<p>Note also that an ACL pertains only to a specific znode. In
+ particular it does not apply to children. For example, if
+ <em>/app</em> is only readable by ip:172.16.16.1 and
+ <em>/app/status</em> is world readable, anyone will
+ be able to read <em>/app/status</em>; ACLs are not
+ recursive.</p>
<p>ZooKeeper supports pluggable authentication schemes. Ids are
specified using the form <em>scheme:id</em>,
where <em>scheme</em> is a the authentication scheme
that the id corresponds to. For
- example, <em>host:host1.corp.com</em> is an id for a
- host named <em>host1.corp.com</em>.</p>
+ example, <em>ip:172.16.16.1</em> is an id for a
+ host with the address <em>172.16.16.1</em>.</p>
<p>When a client connects to ZooKeeper and authenticates
itself, ZooKeeper associates all the ids that correspond to a
client with the clients connection. These ids are checked against
@@ -1028,7 +1037,7 @@
example, the pair <em>(ip:19.22.0.0/16, READ)</em>
gives the <em>READ</em> permission to any clients with
an IP address that starts with 19.22.</p>
-<a name="N10277"></a><a name="sc_ACLPermissions"></a>
+<a name="N10283"></a><a name="sc_ACLPermissions"></a>
<h3 class="h4">ACL Permissions</h3>
<p>ZooKeeper supports the following permissions:</p>
<ul>
@@ -1084,7 +1093,7 @@
node, but nothing more. (The problem is, if you want to call
zoo_exists() on a node that doesn't exist, there is no
permission to check.)</p>
-<a name="N102CD"></a><a name="sc_BuiltinACLSchemes"></a>
+<a name="N102D9"></a><a name="sc_BuiltinACLSchemes"></a>
<h4>Builtin ACL Schemes</h4>
<p>ZooKeeeper has the following built in schemes:</p>
<ul>
@@ -1122,18 +1131,6 @@
<li>
<p>
-<strong>host</strong> uses the
- client host name as an ACL ID identity. The ACL expression is
- a hostname suffix. For example, the ACL
- expression <em>host:corp.com</em> matches the
- ids <em>host:host1.corp.com</em>
- and <em>host:host2.corp.com</em>, but
- not <em>host:host1.store.com</em>.</p>
-</li>
-
-
-<li>
-<p>
<strong>ip</strong> uses the
client host IP as an ACL ID identity. The ACL expression is of
the form <em>addr/bits</em> where the most
@@ -1145,7 +1142,7 @@
</ul>
-<a name="N10323"></a><a name="ZooKeeper+C+client+API"></a>
+<a name="N1031D"></a><a name="ZooKeeper+C+client+API"></a>
<h4>ZooKeeper C client API</h4>
<p>The following constants are provided by the ZooKeeper C
library:</p>
@@ -1367,7 +1364,93 @@
</div>
-<a name="N1043A"></a><a name="ch_zkGuarantees"></a>
+<a name="N10434"></a><a name="sc_ZooKeeperPluggableAuthentication"></a>
+<h2 class="h3">Pluggable ZooKeeper authentication</h2>
+<div class="section">
+<p>ZooKeeper runs in a variety of different environments with
+ various different authentication schemes, so it has a completely
+ pluggable authentication framework. Even the builtin authentication
+ schemes use the pluggable authentication framework.</p>
+<p>To understand how the authentication framework works, first you must
+ understand the two main authentication operations. The framework
+ first must authenticate the client. This is usually done as soon as
+ the client connects to a server and consists of validating information
+ sent from or gathered about a client and associating it with the connection.
+ The second operation handled by the framework is finding the entries in an
+ ACL that correspond to client. ACL entries are <<em>idspec,
+ permissions</em>> pairs. The <em>idspec</em> may be
+ a simple string match against the authentication information associated
+ with the connection or it may be a expression that is evaluated against that
+ information. It is up to the implementation of the authentication plugin
+ to do the match. Here is the interface that an authentication plugin must
+ implement:</p>
+<pre class="code">
+public interface AuthenticationProvider {
+ String getScheme();
+ KeeperException.Code handleAuthentication(ServerCnxn cnxn, byte authData[]);
+ boolean isValid(String id);
+ boolean matches(String id, String aclExpr);
+ boolean isAuthenticated();
+}
+ </pre>
+<p>The first method <em>getScheme</em> returns the string
+ that identifies the plugin. Because we support multiple methods of authentication,
+ an authentication credential or an <em>idspec</em> will always be
+ prefixed with <em>scheme:</em>. The ZooKeeper server uses the scheme
+ returned by the authentication plugin to determine which ids the scheme
+ applies to.</p>
+<p>
+<em>handleAuthentication</em> is called when a client
+ sends authentication information to be associated with a connection. The
+ client specifies the scheme to which the information corresponds. The
+ ZooKeeper server passes the information to the authentication plugin whose
+ <em>getScheme</em> matches the scheme passed by the client. The
+ implementor of <em>handleAuthentication</em> will usually return
+ an error if it determines that the information is bad, or it will associate information
+ with the connection using <em>cnxn.getAuthInfo().add(new Id(getScheme(), data))</em>.
+ </p>
+<p>The authentication plugin is involved in both setting and using ACLs. When an
+ ACL is set for a znode, the ZooKeeper server will pass the id part of the entry to
+ the <em>isValid(String id)</em> method. It is up to the plugin to verify
+ that the id has a correct form. For example, <em>ip:172.16.0.0/16</em>
+ is a valid id, but <em>ip:host.com</em> is not. If the new ACL includes
+ an "auth" entry, <em>isAuthenticated</em> is used to see if the
+ authentication information for this scheme that is assocatied with the connection
+ should be added to the ACL. Some schemes
+ should not be included in auth. For example, the IP address of the client is not
+ considered as an id that should be added to the ACL if auth is specified.</p>
+<p>ZooKeeper invokes
+ <em>matches(String id, String aclExpr)</em> when checking an ACL. It
+ needs to match authentication information of the client against the relevant ACL
+ entries. To find the entries which apply to the client, the ZooKeeper server will
+ find the scheme of each entry and if there is authentication information
+ from that client for that scheme, <em>matches(String id, String aclExpr)</em>
+ will be called with <em>id</em> set to the authentication information
+ that was previously added to the connection by <em>handleAuthentication</em> and
+ <em>aclExpr</em> set to the id of the ACL entry. The authentication plugin
+ uses its own logic and matching scheme to determine if <em>id</em> is included
+ in <em>aclExpr</em>.
+ </p>
+<p>There are two built in authentication plugins: <em>ip</em> and
+ <em>digest</em>. Additional plugins can adding using system properties. At
+ startup the ZooKeeper server will look for system properties that start with
+ "zookeeper.authProvider." and interpret the value of those properties as the class name
+ of an authentication plugin. These properties can be set using the
+ <em>-Dzookeeeper.authProvider.X=com.f.MyAuth</em> or adding entries such as
+ the following in the server configuration file:</p>
+<pre class="code">
+authProvider.1=com.f.MyAuth
+authProvider.2=com.f.MyAuth2
+ </pre>
+<p>Care should be taking to ensure that the suffix on the property is unique. If there are
+ duplicates such as <em>-Dzookeeeper.authProvider.X=com.f.MyAuth -Dzookeeper.authProvider.X=com.f.MyAuth2</em>,
+ only one will be used. Also all servers must have the same plugins defined, otherwise clients using
+ the authentication schemes provided by the plugins will have problems connecting to some servers.
+ </p>
+</div>
+
+
+<a name="N104A0"></a><a name="ch_zkGuarantees"></a>
<h2 class="h3">Consistency Guarantees</h2>
<div class="section">
<p>ZooKeeper is a high performance, scalable service. Both reads and
@@ -1493,12 +1576,12 @@
</div>
-<a name="N104A1"></a><a name="ch_bindings"></a>
+<a name="N10507"></a><a name="ch_bindings"></a>
<h2 class="h3">Bindings</h2>
<div class="section">
<p>The ZooKeeper client libraries come in two languages: Java and C.
The following sections describe these.</p>
-<a name="N104AA"></a><a name="Java+Binding"></a>
+<a name="N10510"></a><a name="Java+Binding"></a>
<h3 class="h4">Java Binding</h3>
<p>There are two packages that make up the ZooKeeper Java binding:
<strong>org.apache.zookeeper</strong> and <strong>org.apache.zookeeper.data</strong>. The rest of the
@@ -1565,7 +1648,7 @@
(SESSION_EXPIRED and AUTH_FAILED), the ZooKeeper object becomes invalid.
On a close, the two threads shut down and any further access on zookeeper
handle is undefined behavior and should be avoided. </p>
-<a name="N104F3"></a><a name="C+Binding"></a>
+<a name="N10559"></a><a name="C+Binding"></a>
<h3 class="h4">C Binding</h3>
<p>The C binding has a single-threaded and multi-threaded library.
The multi-threaded library is easiest to use and is most similar to the
@@ -1582,7 +1665,7 @@
(i.e. FreeBSD 4.x). In all other cases, application developers should
link with zookeeper_mt, as it includes support for both Sync and Async
API.</p>
-<a name="N10502"></a><a name="Installation"></a>
+<a name="N10568"></a><a name="Installation"></a>
<h4>Installation</h4>
<p>If you're building the client from a check-out from the Apache
repository, follow the steps outlined below. If you're building from a
@@ -1713,7 +1796,7 @@
</li>
</ol>
-<a name="N105AB"></a><a name="Using+the+C+Client"></a>
+<a name="N10611"></a><a name="Using+the+C+Client"></a>
<h4>Using the C Client</h4>
<p>You can test your client by running a ZooKeeper server (see
instructions on the project wiki page on how to run it) and connecting
@@ -1771,7 +1854,7 @@
</div>
-<a name="N105F1"></a><a name="ch_guideToZkOperations"></a>
+<a name="N10657"></a><a name="ch_guideToZkOperations"></a>
<h2 class="h3">Building Blocks: A Guide to ZooKeeper Operations</h2>
<div class="section">
<p>This section surveys all the operations a developer can perform
@@ -1789,28 +1872,28 @@
</li>
</ul>
-<a name="N10605"></a><a name="sc_errorsZk"></a>
+<a name="N1066B"></a><a name="sc_errorsZk"></a>
<h3 class="h4">Handling Errors</h3>
<p>Both the Java and C client bindings may report errors. The Java client binding does so by throwing KeeperException, calling code() on the exception will return the specific error code. The C client binding returns an error code as defined in the enum ZOO_ERRORS. API callbacks indicate result code for both language bindings. See the API documentation (javadoc for Java, doxygen for C) for full details on the possible errors and their meaning.</p>
-<a name="N1060F"></a><a name="sc_connectingToZk"></a>
+<a name="N10675"></a><a name="sc_connectingToZk"></a>
<h3 class="h4">Connecting to ZooKeeper</h3>
<p></p>
-<a name="N10618"></a><a name="sc_readOps"></a>
+<a name="N1067E"></a><a name="sc_readOps"></a>
<h3 class="h4">Read Operations</h3>
<p></p>
-<a name="N10621"></a><a name="sc_writeOps"></a>
+<a name="N10687"></a><a name="sc_writeOps"></a>
<h3 class="h4">Write Operations</h3>
<p></p>
-<a name="N1062A"></a><a name="sc_handlingWatches"></a>
+<a name="N10690"></a><a name="sc_handlingWatches"></a>
<h3 class="h4">Handling Watches</h3>
<p></p>
-<a name="N10633"></a><a name="sc_miscOps"></a>
+<a name="N10699"></a><a name="sc_miscOps"></a>
<h3 class="h4">Miscelleaneous ZooKeeper Operations</h3>
<p></p>
</div>
-<a name="N1063D"></a><a name="ch_programStructureWithExample"></a>
+<a name="N106A3"></a><a name="ch_programStructureWithExample"></a>
<h2 class="h3">Program Structure, with Simple Example</h2>
<div class="section">
<p>
@@ -1819,7 +1902,7 @@
</div>
-<a name="N10648"></a><a name="ch_gotchas"></a>
+<a name="N106AE"></a><a name="ch_gotchas"></a>
<h2 class="h3">Gotchas: Common Problems and Troubleshooting</h2>
<div class="section">
<p>So now you know ZooKeeper. It's fast, simple, your application
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf?rev=787789&r1=787788&r2=787789&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml?rev=787789&r1=787788&r2=787789&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml Tue Jun 23 19:03:45 2009
@@ -585,12 +585,19 @@
znode. Instead, an ACL specifies sets of ids and permissions that
are associated with those ids.</para>
+ <para>Note also that an ACL pertains only to a specific znode. In
+ particular it does not apply to children. For example, if
+ <emphasis>/app</emphasis> is only readable by ip:172.16.16.1 and
+ <emphasis>/app/status</emphasis> is world readable, anyone will
+ be able to read <emphasis>/app/status</emphasis>; ACLs are not
+ recursive.</para>
+
<para>ZooKeeper supports pluggable authentication schemes. Ids are
specified using the form <emphasis>scheme:id</emphasis>,
where <emphasis>scheme</emphasis> is a the authentication scheme
that the id corresponds to. For
- example, <emphasis>host:host1.corp.com</emphasis> is an id for a
- host named <emphasis>host1.corp.com</emphasis>.</para>
+ example, <emphasis>ip:172.16.16.1</emphasis> is an id for a
+ host with the address <emphasis>172.16.16.1</emphasis>.</para>
<para>When a client connects to ZooKeeper and authenticates
itself, ZooKeeper associates all the ids that correspond to a
@@ -669,14 +676,6 @@
password <emphasis>digest</emphasis>.</para>
</listitem>
- <listitem><para><emphasis role="bold">host</emphasis> uses the
- client host name as an ACL ID identity. The ACL expression is
- a hostname suffix. For example, the ACL
- expression <emphasis>host:corp.com</emphasis> matches the
- ids <emphasis>host:host1.corp.com</emphasis>
- and <emphasis>host:host2.corp.com</emphasis>, but
- not <emphasis>host:host1.store.com</emphasis>.</para></listitem>
-
<listitem><para><emphasis role="bold">ip</emphasis> uses the
client host IP as an ACL ID identity. The ACL expression is of
the form <emphasis>addr/bits</emphasis> where the most
@@ -859,6 +858,99 @@
</section>
</section>
+ <section id="sc_ZooKeeperPluggableAuthentication">
+ <title>Pluggable ZooKeeper authentication</title>
+
+ <para>ZooKeeper runs in a variety of different environments with
+ various different authentication schemes, so it has a completely
+ pluggable authentication framework. Even the builtin authentication
+ schemes use the pluggable authentication framework.</para>
+
+ <para>To understand how the authentication framework works, first you must
+ understand the two main authentication operations. The framework
+ first must authenticate the client. This is usually done as soon as
+ the client connects to a server and consists of validating information
+ sent from or gathered about a client and associating it with the connection.
+ The second operation handled by the framework is finding the entries in an
+ ACL that correspond to client. ACL entries are <<emphasis>idspec,
+ permissions</emphasis>> pairs. The <emphasis>idspec</emphasis> may be
+ a simple string match against the authentication information associated
+ with the connection or it may be a expression that is evaluated against that
+ information. It is up to the implementation of the authentication plugin
+ to do the match. Here is the interface that an authentication plugin must
+ implement:</para>
+
+ <programlisting>
+public interface AuthenticationProvider {
+ String getScheme();
+ KeeperException.Code handleAuthentication(ServerCnxn cnxn, byte authData[]);
+ boolean isValid(String id);
+ boolean matches(String id, String aclExpr);
+ boolean isAuthenticated();
+}
+ </programlisting>
+
+ <para>The first method <emphasis>getScheme</emphasis> returns the string
+ that identifies the plugin. Because we support multiple methods of authentication,
+ an authentication credential or an <emphasis>idspec</emphasis> will always be
+ prefixed with <emphasis>scheme:</emphasis>. The ZooKeeper server uses the scheme
+ returned by the authentication plugin to determine which ids the scheme
+ applies to.</para>
+
+ <para><emphasis>handleAuthentication</emphasis> is called when a client
+ sends authentication information to be associated with a connection. The
+ client specifies the scheme to which the information corresponds. The
+ ZooKeeper server passes the information to the authentication plugin whose
+ <emphasis>getScheme</emphasis> matches the scheme passed by the client. The
+ implementor of <emphasis>handleAuthentication</emphasis> will usually return
+ an error if it determines that the information is bad, or it will associate information
+ with the connection using <emphasis>cnxn.getAuthInfo().add(new Id(getScheme(), data))</emphasis>.
+ </para>
+
+ <para>The authentication plugin is involved in both setting and using ACLs. When an
+ ACL is set for a znode, the ZooKeeper server will pass the id part of the entry to
+ the <emphasis>isValid(String id)</emphasis> method. It is up to the plugin to verify
+ that the id has a correct form. For example, <emphasis>ip:172.16.0.0/16</emphasis>
+ is a valid id, but <emphasis>ip:host.com</emphasis> is not. If the new ACL includes
+ an "auth" entry, <emphasis>isAuthenticated</emphasis> is used to see if the
+ authentication information for this scheme that is assocatied with the connection
+ should be added to the ACL. Some schemes
+ should not be included in auth. For example, the IP address of the client is not
+ considered as an id that should be added to the ACL if auth is specified.</para>
+
+ <para>ZooKeeper invokes
+ <emphasis>matches(String id, String aclExpr)</emphasis> when checking an ACL. It
+ needs to match authentication information of the client against the relevant ACL
+ entries. To find the entries which apply to the client, the ZooKeeper server will
+ find the scheme of each entry and if there is authentication information
+ from that client for that scheme, <emphasis>matches(String id, String aclExpr)</emphasis>
+ will be called with <emphasis>id</emphasis> set to the authentication information
+ that was previously added to the connection by <emphasis>handleAuthentication</emphasis> and
+ <emphasis>aclExpr</emphasis> set to the id of the ACL entry. The authentication plugin
+ uses its own logic and matching scheme to determine if <emphasis>id</emphasis> is included
+ in <emphasis>aclExpr</emphasis>.
+ </para>
+
+ <para>There are two built in authentication plugins: <emphasis>ip</emphasis> and
+ <emphasis>digest</emphasis>. Additional plugins can adding using system properties. At
+ startup the ZooKeeper server will look for system properties that start with
+ "zookeeper.authProvider." and interpret the value of those properties as the class name
+ of an authentication plugin. These properties can be set using the
+ <emphasis>-Dzookeeeper.authProvider.X=com.f.MyAuth</emphasis> or adding entries such as
+ the following in the server configuration file:</para>
+
+ <programlisting>
+authProvider.1=com.f.MyAuth
+authProvider.2=com.f.MyAuth2
+ </programlisting>
+
+ <para>Care should be taking to ensure that the suffix on the property is unique. If there are
+ duplicates such as <emphasis>-Dzookeeeper.authProvider.X=com.f.MyAuth -Dzookeeper.authProvider.X=com.f.MyAuth2</emphasis>,
+ only one will be used. Also all servers must have the same plugins defined, otherwise clients using
+ the authentication schemes provided by the plugins will have problems connecting to some servers.
+ </para>
+ </section>
+
<section id="ch_zkGuarantees">
<title>Consistency Guarantees</title>