You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by ph...@apache.org on 2008/10/04 01:35:28 UTC
svn commit: r701567 [1/2] - in /hadoop/zookeeper/trunk: ./ docs/ docs/skin/
src/docs/src/documentation/ src/docs/src/documentation/content/xdocs/
Author: phunt
Date: Fri Oct 3 16:35:27 2008
New Revision: 701567
URL: http://svn.apache.org/viewvc?rev=701567&view=rev
Log:
ZOOKEEPER-156. update programmer guide with acl details from old wiki page
Modified:
hadoop/zookeeper/trunk/CHANGES.txt
hadoop/zookeeper/trunk/docs/recipes.html
hadoop/zookeeper/trunk/docs/recipes.pdf
hadoop/zookeeper/trunk/docs/skin/profile.css
hadoop/zookeeper/trunk/docs/zookeeperAdmin.html
hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf
hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.html
hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.pdf
hadoop/zookeeper/trunk/docs/zookeeperOver.html
hadoop/zookeeper/trunk/docs/zookeeperOver.pdf
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
hadoop/zookeeper/trunk/docs/zookeeperStarted.html
hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOtherInfo.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/skinconf.xml
Modified: hadoop/zookeeper/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/CHANGES.txt?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/CHANGES.txt (original)
+++ hadoop/zookeeper/trunk/CHANGES.txt Fri Oct 3 16:35:27 2008
@@ -17,6 +17,9 @@
BUGFIXES:
+ ZOOKEEPER-156. update programmer guide with acl details from old wiki page
+ (phunt)
+
ZOOKEEPER-154. reliability graph diagram in overview doc needs context
(phunt)
Modified: hadoop/zookeeper/trunk/docs/recipes.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/recipes.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/recipes.html (original)
+++ hadoop/zookeeper/trunk/docs/recipes.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper Recipes and Solutions</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,10 +157,11 @@
<a class="dida" href="recipes.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper Recipes and Solutions</h1>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#A+Guide+to+Creating+Higher-level+Constructs+with+ZooKeeper">A Guide to Creating Higher-level Constructs with ZooKeeper</a>
+<a href="#ch_recipes">A Guide to Creating Higher-level Constructs with ZooKeeper</a>
<ul class="minitoc">
<li>
<a href="#sc_outOfTheBox">Out of the Box Applications: Name Service, Configuration, Group
@@ -204,12 +205,11 @@
</ul>
</div>
-<title>ZooKeeper Recipes and Solutions</title>
-<a name="N1000A"></a><a name="A+Guide+to+Creating+Higher-level+Constructs+with+ZooKeeper"></a>
+<a name="N10009"></a><a name="ch_recipes"></a>
<h2 class="h3">A Guide to Creating Higher-level Constructs with ZooKeeper</h2>
<div class="section">
<p>In this article, you'll find guidelines for using
@@ -234,7 +234,7 @@
as event handles or queues, a more practical means of performing the same
function. In general, the examples in this section are designed to
stimulate thought.</p>
-<a name="N10022"></a><a name="sc_outOfTheBox"></a>
+<a name="N10021"></a><a name="sc_outOfTheBox"></a>
<h3 class="h4">Out of the Box Applications: Name Service, Configuration, Group
Membership</h3>
<p>Name service and configuration are two of the primary applications
@@ -245,13 +245,14 @@
group create ephemeral nodes under the group node. Nodes of the members
that fail abnormally will be removed automatically when ZooKeeper detects
the failure.</p>
-<a name="N10032"></a><a name="sc_recipes_eventHandles"></a>
+<a name="N10031"></a><a name="sc_recipes_eventHandles"></a>
<h3 class="h4">Barriers</h3>
-<p>Distributed systems use <em>barriers</em> to block
- processing of a set of nodes until a condition is met at which time all
- the nodes are allowed to proceed. Barriers are implemented in ZooKeeper by
- designating a barrier node. The barrier is in place if the barrier node
- exists. Here's the pseudo code:</p>
+<p>Distributed systems use <em>barriers</em>
+ to block processing of a set of nodes until a condition is met
+ at which time all the nodes are allowed to proceed. Barriers are
+ implemented in ZooKeeper by designating a barrier node. The
+ barrier is in place if the barrier node exists. Here's the
+ pseudo code:</p>
<ol>
<li>
@@ -288,7 +289,7 @@
</li>
</ol>
-<a name="N10068"></a><a name="sc_doubleBarriers"></a>
+<a name="N10067"></a><a name="sc_doubleBarriers"></a>
<h4>Double Barriers</h4>
<p>Double barriers enable clients to synchronize the beginning and
the end of a computation. When enough processes have joined the barrier,
@@ -302,143 +303,144 @@
<em>x</em> client process register before proceeding with
the computation. (The <em>x</em> here is up to you to
determine for your system.)</p>
-<p>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
-
-
+
+
<tr>
-
+
<td><strong>Enter</strong></td>
- <td><strong>Leave</strong></td>
-
+ <td><strong>Leave</strong></td>
+
</tr>
-
+
<tr>
-
+
<td>
<ol>
-
+
<li>
-
+
<p>Create a name <em><em>n</em> =
- <em>b</em>+“/”+<em>p</em></em>
+ <em>b</em>+“/”+<em>p</em></em>
</p>
-
+
</li>
-
+
<li>
-
+
<p>Set watch: <strong>exists(<em>b</em> + ‘‘/ready’’,
- true)</strong>
+ true)</strong>
</p>
-
+
</li>
-
+
<li>
-
+
<p>Create child: <strong>create(
- <em>n</em>, EPHEMERAL)</strong>
+ <em>n</em>, EPHEMERAL)</strong>
</p>
-
+
</li>
-
+
<li>
-
+
<p>
<strong>L = getChildren(b,
- false)</strong>
+ false)</strong>
</p>
-
+
</li>
-
+
<li>
-
-<p>if fewer children in L than<em>
- x</em>, wait for watch event</p>
+<p>if fewer children in L than<em>
+ x</em>, wait for watch event</p>
+
</li>
-
+
<li>
-
+
<p>else <strong>create(b + ‘‘/ready’’,
- REGULAR)</strong>
+ REGULAR)</strong>
</p>
-
-</li>
+</li>
+
</ol>
</td>
- <td>
+ <td>
<ol>
-
+
<li>
-
+
<p>
<strong>L = getChildren(b,
- false)</strong>
+ false)</strong>
</p>
-
+
</li>
-
+
<li>
-
-<p>if no children, exit</p>
+<p>if no children, exit</p>
+
</li>
-
+
<li>
-
+
<p>if <em>p</em> is only process node in
L, delete(n) and exit</p>
-
+
</li>
-
+
<li>
-
+
<p>if <em>p</em> is the lowest process
node in L, wait on highest process node in P</p>
-
+
</li>
-
+
<li>
-
+
<p>else <strong>delete(<em>n</em>) </strong>if
still exists and wait on lowest process node in L</p>
-
+
</li>
-
+
<li>
-
-<p>goto 1</p>
-</li>
+<p>goto 1</p>
+</li>
+
</ol>
</td>
-
-</tr>
-
-</table>On entering, all processes watch on a ready node and
- create an ephemeral node as a child of the barrier node. Each process
- but the last enters the barrier and waits for the ready node to appear
- at line 5. The process that creates the xth node, the last process, will
- see x nodes in the list of children and create the ready node, waking up
- the other processes. Note that waiting processes wake up only when it is
- time to exit, so waiting is efficient.</p>
+</tr>
+
+
+</table>
+<p>On entering, all processes watch on a ready node and
+ create an ephemeral node as a child of the barrier node. Each process
+ but the last enters the barrier and waits for the ready node to appear
+ at line 5. The process that creates the xth node, the last process, will
+ see x nodes in the list of children and create the ready node, waking up
+ the other processes. Note that waiting processes wake up only when it is
+ time to exit, so waiting is efficient.
+ </p>
<p>On exit, you can't use a flag such as <em>ready</em>
because you are watching for process nodes to go away. By using
ephemeral nodes, processes that fail after the barrier has been entered
@@ -595,7 +597,7 @@
<p>Call <strong>create( )</strong> to
create a node with pathname
- "<span class="codefrag parameter">_locknode_/read-</span>". This is the
+ "<span class="codefrag filename">_locknode_/read-</span>". This is the
lock node use later in the protocol. Make sure to set both
the <em>sequence</em> and
<em>ephemeral</em> flags.</p>
@@ -616,7 +618,7 @@
<li>
<p>If there are no children with a pathname starting
- with "<span class="codefrag parameter">write-</span>" and having a lower
+ with "<span class="codefrag filename">write-</span>" and having a lower
sequence number than the node created in step <strong>1</strong>, the client has the lock and can
exit the protocol. </p>
@@ -628,7 +630,7 @@
<p>Otherwise, call <strong>exists(
)</strong>, with <em>watch</em> flag, set on
the node in lock directory with pathname staring with
- "<span class="codefrag parameter">write-</span>" having the next lowest
+ "<span class="codefrag filename">write-</span>" having the next lowest
sequence number.</p>
</li>
@@ -660,7 +662,7 @@
<p>Call <strong>create( )</strong> to
create a node with pathname
- "<span class="codefrag parameter">_locknode_/write-</span>". This is the
+ "<span class="codefrag filename">_locknode_/write-</span>". This is the
lock node spoken of later in the protocol. Make sure to
set both <em>sequence</em> and
<em>ephemeral</em> flags.</p>
@@ -713,24 +715,24 @@
</table>
-<p>
<div class="note">
<div class="label">Note</div>
<div class="content">
-
-<p>It might appear that this recipe creates a herd effect: when
- there is a large group of clients waiting for a read lock, and all
- getting notified more or less simultaneously when the
- "<span class="codefrag parameter">write-</span>" node with the lowest sequence number
- is deleted. In fact. that's valid behavior: as all those waiting
- reader clients should be released since they have the lock. The herd
- effect refers to releasing a "herd" when in fact only a single or a
- small number of machines can proceed.</p>
+<p>It might appear that this recipe creates a herd effect:
+ when there is a large group of clients waiting for a read
+ lock, and all getting notified more or less simultaneously
+ when the "<span class="codefrag filename">write-</span>" node with the lowest
+ sequence number is deleted. In fact. that's valid behavior:
+ as all those waiting reader clients should be released since
+ they have the lock. The herd effect refers to releasing a
+ "herd" when in fact only a single or a small number of
+ machines can proceed.
+ </p>
+
</div>
</div>
-</p>
-<a name="N10280"></a><a name="sc_recoverableSharedLocks"></a>
+<a name="N1027F"></a><a name="sc_recoverableSharedLocks"></a>
<h4>Recoverable Shared Locks</h4>
<p>With minor modifications to the Shared Lock protocol, you make
shared locks revocable by modifying the shared lock protocol:</p>
@@ -752,7 +754,7 @@
Laser Beams</em> by stipulating in your protocol that the revoker
is allowed to delete the lock node if after some length of time the lock
isn't deleted by the lock holder.</p>
-<a name="N102AC"></a><a name="sc_recipes_twoPhasedCommit"></a>
+<a name="N102AB"></a><a name="sc_recipes_twoPhasedCommit"></a>
<h3 class="h4">Two-phased Commit</h3>
<p>A two-phase commit protocol is an algorithm that lets all clients in
a distributed system agree either to commit a transaction or abort.</p>
@@ -784,7 +786,7 @@
<p>To address the second problem, you can have the coordinator
propagate the transaction to the sites, and have each site creating its
own ephemeral node.</p>
-<a name="N102C5"></a><a name="sc_leaderElection"></a>
+<a name="N102C4"></a><a name="sc_leaderElection"></a>
<h3 class="h4">Leader Election</h3>
<p>A simple way of doing leader election with ZooKeeper is to use the
<strong>SEQUENCE|EPHEMERAL</strong> flags when creating
Modified: hadoop/zookeeper/trunk/docs/recipes.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/recipes.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/skin/profile.css
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/skin/profile.css?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/skin/profile.css (original)
+++ hadoop/zookeeper/trunk/docs/skin/profile.css Fri Oct 3 16:35:27 2008
@@ -155,4 +155,9 @@
background-color: #f0f0f0;
font-family: monospace;
}
+
+ .code {
+ font-family: "Courier New", Courier, monospace;
+ font-size: 110%;
+ }
\ No newline at end of file
Modified: hadoop/zookeeper/trunk/docs/zookeeperAdmin.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperAdmin.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperAdmin.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperAdmin.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper Administrator's Guide</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,10 +157,12 @@
<a class="dida" href="zookeeperAdmin.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper Administrator's Guide</h1>
+<h3>A Guide to Deployment and Administration</h3>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#Deployment">Deployment</a>
+<a href="#ch_deployment">Deployment</a>
<ul class="minitoc">
<li>
<a href="#sc_systemReq">System Requirements</a>
@@ -174,7 +176,7 @@
</ul>
</li>
<li>
-<a href="#Administration">Administration</a>
+<a href="#ch_administration">Administration</a>
<ul class="minitoc">
<li>
<a href="#sc_configuration">Configuration Parameters</a>
@@ -224,18 +226,16 @@
</ul>
</div>
-<title>ZooKeeper Administrator's Guide</title>
-<subtitle>A Guide to Deployment and Administration</subtitle>
-<a name="N1000D"></a><a name="Deployment"></a>
+<a name="N1000B"></a><a name="ch_deployment"></a>
<h2 class="h3">Deployment</h2>
<div class="section">
-<p>This chapter contains information about deploying Zookeeper and
+<p>This section contains information about deploying Zookeeper and
covers these topics:</p>
<ul>
@@ -271,14 +271,14 @@
section covers situations in which you are setting up Zookeeper on a
limited basis - for evaluation, testing, or development - but not in a
production environment.</p>
-<a name="N10034"></a><a name="sc_systemReq"></a>
+<a name="N10032"></a><a name="sc_systemReq"></a>
<h3 class="h4">System Requirements</h3>
<p>Zookeeper runs in Java, release 1.5 or greater, as group of hosts
called a quorum. Three Zookeeper hosts per quorum is the minimum
recommended quorum size. At Yahoo!, Zookeeper is usually deployed on
dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB
IDE harddrives.</p>
-<a name="N1003E"></a><a name="sc_zkMulitServerSetup"></a>
+<a name="N1003C"></a><a name="sc_zkMulitServerSetup"></a>
<h3 class="h4">Clustered (Multi-Server) Setup</h3>
<p>For reliable ZooKeeper service, you should deploy ZooKeeper in a
cluster known as a <em>quorum</em>. As long as a majority
@@ -298,7 +298,9 @@
<p>Install the Java JDK:</p>
-<pre class="code">$yinst -i jdk-1.6.0.00_3 -br test </pre>
+<p>
+<span class="codefrag computeroutput">$yinst -i jdk-1.6.0.00_3 -br test </span>
+</p>
</li>
@@ -319,7 +321,9 @@
<p>Install the Zookeeper Server Package:</p>
-<pre class="code">$ yinst install -nostart zookeeper_server </pre>
+<p>
+<span class="codefrag computeroutput">$ yinst install -nostart zookeeper_server </span>
+</p>
</li>
@@ -330,7 +334,8 @@
Use the following settings as a starting point:</p>
-<pre class="code">
+<p>
+<span class="codefrag computeroutput">
tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181
@@ -338,7 +343,8 @@
syncLimit=2
server.1=zoo1:2888
server.2=zoo2:2888
-server.3=zoo3:2888</pre>
+server.3=zoo3:2888</span>
+</p>
<p>You can find the meanings of these and other configuration
@@ -367,8 +373,10 @@
Zookeeper:</p>
-<pre class="code">$ java -cp zookeeper-dev.jar:java/lib/log4j-1.2.15.jar:conf \
- org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg</pre>
+<p>
+<span class="codefrag computeroutput">$ java -cp zookeeper-dev.jar:java/lib/log4j-1.2.15.jar:conf \
+ org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg</span>
+</p>
</li>
@@ -386,8 +394,10 @@
simple operations:</p>
-<pre class="code">$ java -cp zookeeper.jar:java/lib/log4j-1.2.15.jar:conf \
- org.apache.zookeeper.ZooKeeperMain 127.0.0.1:2181</pre>
+<p>
+<span class="codefrag computeroutput">$ java -cp zookeeper.jar:java/lib/log4j-1.2.15.jar:conf \
+ org.apache.zookeeper.ZooKeeperMain 127.0.0.1:2181</span>
+</p>
</li>
@@ -400,13 +410,17 @@
client:</p>
-<pre class="code">$ _make cli_st_</pre>
+<p>
+<span class="codefrag computeroutput">$ _make cli_st_</span>
+</p>
<p>And this compiles the mulithreaded client:</p>
-<pre class="code">$ _make cli_mt_</pre>
+<p>
+<span class="codefrag computeroutput">$ _make cli_mt_</span>
+</p>
</li>
@@ -418,12 +432,14 @@
client, for example, you would run:</p>
-<pre class="code">$ cli_mt 127.0.0.1:2181</pre>
+<p>
+<span class="codefrag computeroutput">$ cli_mt 127.0.0.1:2181</span>
+</p>
</li>
</ol>
-<a name="N100C0"></a><a name="sc_singleAndDevSetup"></a>
+<a name="N100CE"></a><a name="sc_singleAndDevSetup"></a>
<h3 class="h4">Single Server and Developer Setup</h3>
<p>If you want to setup Zookeeper for development purposes, you will
probably want to setup a single server instance of Zookeeper, and then
@@ -442,11 +458,12 @@
</div>
-<a name="N100E1"></a><a name="Administration"></a>
+<a name="N100EF"></a><a name="ch_administration"></a>
<h2 class="h3">Administration</h2>
<div class="section">
-<p>This chapter contains information about running and maintaining
- ZooKeeper and covers these topics: <ul>
+<p>This section contains information about running and maintaining
+ ZooKeeper and covers these topics: </p>
+<ul>
<li>
@@ -493,8 +510,7 @@
</li>
</ul>
-</p>
-<a name="N10114"></a><a name="sc_configuration"></a>
+<a name="N10123"></a><a name="sc_configuration"></a>
<h3 class="h4">Configuration Parameters</h3>
<p>ZooKeeper's behavior is governed by the ZooKeeper configuration
file. This file is designed so that the exact same file can be used by
@@ -502,7 +518,7 @@
layouts are the same. If servers use different configuration files,
care must be taken to ensure that the list of servers in all of the
different configuration files match.</p>
-<a name="N1011D"></a><a name="sc_minimumConfiguration"></a>
+<a name="N1012C"></a><a name="sc_minimumConfiguration"></a>
<h4>Minimum Configuration</h4>
<p>Here are the minimum configuration keywords that must be
defined in the configuration file:</p>
@@ -551,7 +567,7 @@
</dl>
-<a name="N10144"></a><a name="sc_advancedConfiguration"></a>
+<a name="N10153"></a><a name="sc_advancedConfiguration"></a>
<h4>Advanced Configuration</h4>
<p>The configuration settings in the section are optional. You
can use them to further fine tune the behaviour of your Zookeeper
@@ -645,7 +661,7 @@
</dl>
-<a name="N101A4"></a><a name="sc_clusterOptions"></a>
+<a name="N101B3"></a><a name="sc_clusterOptions"></a>
<h4>Cluster Options</h4>
<p>The options in this section are designed for use in quorums --
that is, when deploying clusters of servers.</p>
@@ -749,7 +765,7 @@
</dl>
<p></p>
-<a name="N10210"></a><a name="Unsafe+Options"></a>
+<a name="N1021F"></a><a name="Unsafe+Options"></a>
<h4>Unsafe Options</h4>
<p>The following options can be useful, but be careful when you
use them. The risk of each is explained along with the explanation
@@ -799,7 +815,7 @@
</dl>
-<a name="N10242"></a><a name="sc_zkCommands"></a>
+<a name="N10251"></a><a name="sc_zkCommands"></a>
<h3 class="h4">Zookeeper Commands: The Four Letter Words</h3>
<p>Zookeeper responds to a small set of commands. Each command is composed of
four letters. You issue the commands to Zookeeper via telnet or nc, at
@@ -846,14 +862,18 @@
</dl>
<p>Here's an example of the <strong>ruok</strong>
command:</p>
-<pre class="code">$ echo ruok | nc 127.0.0.1 5111
+<p>
+<span class="codefrag computeroutput">$ echo ruok | nc 127.0.0.1 5111
-imok
-</pre>
-<a name="N10274"></a><a name="sc_monitoring"></a>
+imok</span>
+
+</p>
+<a name="N10286"></a><a name="sc_monitoring"></a>
<h3 class="h4">Monitoring</h3>
-<remark>[tbd]</remark>
-<a name="N1027E"></a><a name="sc_dataFileManagement"></a>
+<p>
+<em>[tbd]</em>
+</p>
+<a name="N10291"></a><a name="sc_dataFileManagement"></a>
<h3 class="h4">Data File Management</h3>
<p>ZooKeeper stores its data in a data directory and its transaction
log in a transaction log directory. By default these two directories are
@@ -861,7 +881,7 @@
transaction log files in a separate directory than the data files.
Throughput increases and latency decreases when transaction logs reside
on a dedicated log devices.</p>
-<a name="N10287"></a><a name="The+Data+Directory"></a>
+<a name="N1029A"></a><a name="The+Data+Directory"></a>
<h4>The Data Directory</h4>
<p>This directory has two files in it:</p>
<ul>
@@ -907,14 +927,14 @@
idempotent nature of its updates. By replaying the transaction log
against fuzzy snapshots ZooKeeper gets the state of the system at the
end of the log.</p>
-<a name="N102C3"></a><a name="The+Log+Directory"></a>
+<a name="N102D6"></a><a name="The+Log+Directory"></a>
<h4>The Log Directory</h4>
<p>The Log Directory contains the ZooKeeper transaction logs.
Before any update takes place, ZooKeeper ensures that the transaction
that represents the update is written to non-volatile storage. A new
log file is started each time a snapshot is begun. The log file's
suffix is the first zxid written to that log.</p>
-<a name="N102CD"></a><a name="File+Management"></a>
+<a name="N102E0"></a><a name="File+Management"></a>
<h4>File Management</h4>
<p>The format of snapshot and log files does not change between
standalone ZooKeeper servers and different configurations of
@@ -931,7 +951,7 @@
needs the latest complete fuzzy snapshot and the log files from the
start of that snapshot. The PurgeTxnLog utility implements a simple
retention policy that administrators can use.</p>
-<a name="N102DE"></a><a name="sc_commonProblems"></a>
+<a name="N102F1"></a><a name="sc_commonProblems"></a>
<h3 class="h4">Things to Avoid</h3>
<p>Here are some common problems you can avoid by configuring
ZooKeeper correctly:</p>
@@ -985,10 +1005,10 @@
</dd>
</dl>
-<a name="N10302"></a><a name="sc_bestPractices"></a>
+<a name="N10315"></a><a name="sc_bestPractices"></a>
<h3 class="h4">Best Practices</h3>
<p>For best results, take note of the following list of good
- Zookeeper practices. <remark>[tbd...]</remark>
+ Zookeeper practices. <em>[tbd...]</em>
</p>
</div>
Modified: hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,20 +157,20 @@
<a class="dida" href="zookeeperOtherInfo.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper</h1>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#Other+Info">Other Info</a>
+<a href="#ch_placeholder">Other Info</a>
</li>
</ul>
</div>
-<title>ZooKeeper</title>
-<a name="N1000A"></a><a name="Other+Info"></a>
+<a name="N10009"></a><a name="ch_placeholder"></a>
<h2 class="h3">Other Info</h2>
<div class="section">
<p> currently empty </p>
Modified: hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOtherInfo.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperOver.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOver.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperOver.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperOver.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,10 +157,11 @@
<a class="dida" href="zookeeperOver.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper</h1>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#ZooKeeper%3A+A+Distributed+Coordination+Service+for+Distributed%0A++++Applications">ZooKeeper: A Distributed Coordination Service for Distributed
+<a href="#ch_DesignOverview">ZooKeeper: A Distributed Coordination Service for Distributed
Applications</a>
<ul class="minitoc">
<li>
@@ -191,6 +192,9 @@
<a href="#Performance">Performance</a>
</li>
<li>
+<a href="#Reliability">Reliability</a>
+</li>
+<li>
<a href="#The+ZooKeeper+Project">The ZooKeeper Project</a>
</li>
</ul>
@@ -198,12 +202,11 @@
</ul>
</div>
-<title>ZooKeeper</title>
-<a name="N1000A"></a><a name="ZooKeeper%3A+A+Distributed+Coordination+Service+for+Distributed%0A++++Applications"></a>
+<a name="N10009"></a><a name="ch_DesignOverview"></a>
<h2 class="h3">ZooKeeper: A Distributed Coordination Service for Distributed
Applications</h2>
<div class="section">
@@ -218,7 +221,7 @@
especially prone to errors such as race conditions and deadlock. The
motivation behind ZooKeeper is to relieve distributed applications the
responsibility of implementing coordination services from scratch.</p>
-<a name="N10016"></a><a name="sc_designGoals"></a>
+<a name="N10015"></a><a name="sc_designGoals"></a>
<h3 class="h4">Design Goals</h3>
<p>
<strong>ZooKeeper is simple.</strong> ZooKeeper
@@ -271,7 +274,7 @@
especially fast in "read-dominant" workloads. ZooKeeper applications run
on thousands of machines, and it performs best where reads are more
common than writes, at ratios of around 10:1.</p>
-<a name="N10046"></a><a name="sc_dataModelNameSpace"></a>
+<a name="N10045"></a><a name="sc_dataModelNameSpace"></a>
<h3 class="h4">Data model and the hierarchical namespace</h3>
<p>The name space provided by ZooKeeper is much like that of a
standard file system. A name is a sequence of path elements separated by
@@ -289,7 +292,7 @@
</td>
</tr>
</table>
-<a name="N1005C"></a><a name="Nodes+and+ephemeral+nodes"></a>
+<a name="N1005B"></a><a name="Nodes+and+ephemeral+nodes"></a>
<h3 class="h4">Nodes and ephemeral nodes</h3>
<p>Unlike is standard file systems, each node in a ZooKeeper
namespace can have data associated with it as well as children. It is
@@ -311,8 +314,8 @@
<p>ZooKeeper also has the notion of ephemeral nodes. These znodes
exists as long as the session that created the znode is active. When the
session ends the znode is deleted. Ephemeral nodes are useful when you
- want to implement <remark>[tbd]</remark>.</p>
-<a name="N10075"></a><a name="Conditional+updates+and+watches"></a>
+ want to implement <em>[tbd]</em>.</p>
+<a name="N10074"></a><a name="Conditional+updates+and+watches"></a>
<h3 class="h4">Conditional updates and watches</h3>
<p>ZooKeeper supports the concept of <em>watches</em>.
Clients can set a watch on a znodes. A watch will be triggered and
@@ -320,8 +323,8 @@
receives a packet saying that the znode has changed. And if the
connection between the client and one of the Zoo Keeper servers is
broken, the client will receive a local notification. These can be used
- to <remark>[tbd]</remark>.</p>
-<a name="N10085"></a><a name="Guarantees"></a>
+ to <em>[tbd]</em>.</p>
+<a name="N10084"></a><a name="Guarantees"></a>
<h3 class="h4">Guarantees</h3>
<p>ZooKeeper is very fast and very simple. Since its goal, though, is
to be a basis for the construction of more complicated services, such as
@@ -373,9 +376,9 @@
</ul>
<p>For more information on these, and how they can be used, see
- <remark>[tbd]</remark>
+ <em>[tbd]</em>
</p>
-<a name="N100BB"></a><a name="Simple+API"></a>
+<a name="N100BA"></a><a name="Simple+API"></a>
<h3 class="h4">Simple API</h3>
<p>One of the design goals of ZooKeeper is provide a very simple
programming interface. As a result, it supports only these
@@ -440,9 +443,9 @@
</dl>
<p>For a more in-depth discussion on these, and how they can be used
to implement higher level operations, please refer to
- <remark>[tbd]</remark>
+ <em>[tbd]</em>
</p>
-<a name="N100FE"></a><a name="Implementation"></a>
+<a name="N100FD"></a><a name="Implementation"></a>
<h3 class="h4">Implementation</h3>
<p>
<a href="#fg_zkComponents">ZooKeeper Components</a> shows the high-level components
@@ -450,20 +453,18 @@
each of
the servers that make up the ZooKeeper service replicates its own copy
of each of components.</p>
-<p>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
<tr>
<td>ZooKeeper Components</td>
</tr>
<tr>
<td>
-
- <img alt="" src="images/zkcomponents.jpg">
-
- </td>
+
+ <img alt="" src="images/zkcomponents.jpg">
+
+ </td>
</tr>
</table>
-</p>
<p>The replicated database is an in-memory database containing the
entire data tree. Updates are logged to disk for recoverability, and
writes are serialized to disk before they are applied to the in-memory
@@ -486,16 +487,16 @@
calculates what the state of the system is when the write is to be
applied and transforms this into a transaction that captures this new
state.</p>
-<a name="N1012A"></a><a name="Uses"></a>
+<a name="N10128"></a><a name="Uses"></a>
<h3 class="h4">Uses</h3>
<p>The programming interface to ZooKeeper is deliberately simple.
With it, however, you can implement higher order operations, such as
synchronizations primitives, group membership, ownership, etc. Some
- distributed applications have used it to: <remark>[tbd: add uses from
- white paper and video presentation.]</remark> For more information, see
- <remark>[tbd]</remark>
+ distributed applications have used it to: <em>[tbd: add uses from
+ white paper and video presentation.]</em> For more information, see
+ <em>[tbd]</em>
</p>
-<a name="N10139"></a><a name="Performance"></a>
+<a name="N10137"></a><a name="Performance"></a>
<h3 class="h4">Performance</h3>
<p>ZooKeeper is designed to be highly performant. But is it? The
results of the ZooKeeper's development team at Yahoo! Research indicate
@@ -503,19 +504,19 @@
performance in applications where reads outnumber writes, since writes
involve synchronizing the state of all servers. (Reads outnumbering
writes is typically the case for a coordination service.)</p>
-<p>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
<tr>
<td>ZooKeeper Throughput as the Read-Write Ratio Varies</td>
</tr>
<tr>
<td>
-
- <img alt="" src="images/zkperfRW.jpg">
-
- </td>
+
+ <img alt="" src="images/zkperfRW.jpg">
+
+ </td>
</tr>
-</table>Benchmarks also indicate that it is reliable, too. <a href="#fg_zkPerfReliability">Reliability in the Presence of Errors</a> shows how a deployment responds to
+</table>
+<p>Benchmarks also indicate that it is reliable, too. <a href="#fg_zkPerfReliability">Reliability in the Presence of Errors</a> shows how a deployment responds to
various failures. The events marked in the figure are the
following:</p>
<ol>
@@ -555,20 +556,27 @@
</li>
</ol>
-<p>
+<a name="N10179"></a><a name="Reliability"></a>
+<h3 class="h4">Reliability</h3>
+<p>To show the behavior of the system over time as
+ failures are injected we ran a ZooKeeper service made up of
+ 7 machines. We ran the same saturation benchmark as before,
+ but this time we kept the write percentage at a constant
+ 30%, which is a conservative ratio of our expected
+ workloads.
+ </p>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
<tr>
<td>Reliability in the Presence of Errors</td>
</tr>
<tr>
<td>
-
- <img alt="" src="images/zkperfreliability.jpg">
-
- </td>
+
+ <img alt="" src="images/zkperfreliability.jpg">
+
+ </td>
</tr>
</table>
-</p>
<p>The are a few important observations from this graph. First, if
followers fail and recover quickly, then ZooKeeper is able to sustain a
high throughput despite the failure. But maybe more importantly, the
@@ -577,23 +585,28 @@
ZooKeeper takes less than 200ms to elect a new leader. Third, as
followers recover, ZooKeeper is able to raise throughput again once they
start processing requests.</p>
-<a name="N1018A"></a><a name="The+ZooKeeper+Project"></a>
+<a name="N10192"></a><a name="The+ZooKeeper+Project"></a>
<h3 class="h4">The ZooKeeper Project</h3>
-<p>ZooKeeper has been successfully used in industrial applications.
- It is used at Yahoo! as the coordination and failure recovery service
- for Yahoo! Message Broker, which is a highly scalable publish-subscribe
- system managing thousands of topics for replication and data delivery.
- It is used by the Fetching Service for Yahoo! crawler, where it also
- manages failure recovery. And it is used by Hadoop On Demand (HOD),
- which is an open source implementation of the map-reduce model of
- computation. HOD uses Zookeeper as a communications and control channel
- between slave and master process. (For more information, see the <a href="http://hadoop.apache.org/core/">Hadoop</a> and <a href="http://hadoop.apache.org/core/docs/current/hod.html">Hadoop on
- Demand</a> open source projects on Apache.)</p>
-<p>ZooKeeper itself is an open source project, under the Apache Open
- Source Foundation. It is a subproject of Hadoop. All users and
- developers are encourged to join the community and contribute their
- expertise. See the <a href="http://hadoop.apache.org/zookeeper/">Zookeeper Project on
- Apache</a> for more information.</p>
+<p>ZooKeeper has been
+ <a href="http://wiki.apache.org/hadoop/ZooKeeper/PoweredBy">
+ successfully used
+ </a>
+ in many industrial applications. It is used at Yahoo! as the
+ coordination and failure recovery service for Yahoo! Message
+ Broker, which is a highly scalable publish-subscribe system
+ managing thousands of topics for replication and data
+ delivery. It is used by the Fetching Service for Yahoo!
+ crawler, where it also manages failure recovery. A number of
+ Yahoo! advertising systems also use ZooKeeper to implement
+ reliable services.
+ </p>
+<p>All users and developers are encouraged to join the
+ community and contribute their expertise. See the
+ <a href="http://hadoop.apache.org/zookeeper/">
+ Zookeeper Project on Apache
+ </a>
+ for more information.
+ </p>
</div>
<p align="right">
Modified: hadoop/zookeeper/trunk/docs/zookeeperOver.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOver.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper Programmer's Guide</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,10 +157,15 @@
<a class="dida" href="zookeeperProgrammers.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper Programmer's Guide</h1>
+<h3>Developing Distributed Applications that use ZooKeeper</h3>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#The+ZooKeeper+Data+Model">The ZooKeeper Data Model</a>
+<a href="#_introduction">Introduction</a>
+</li>
+<li>
+<a href="#ch_zkDataModel">The ZooKeeper Data Model</a>
<ul class="minitoc">
<li>
<a href="#sc_zkDataModel_znodes">ZNodes</a>
@@ -188,10 +193,10 @@
</ul>
</li>
<li>
-<a href="#ZooKeeper+Sessions">ZooKeeper Sessions</a>
+<a href="#ch_zkSessions">ZooKeeper Sessions</a>
</li>
<li>
-<a href="#ZooKeeper+Watches">ZooKeeper Watches</a>
+<a href="#ch_zkWatches">ZooKeeper Watches</a>
<ul class="minitoc">
<li>
<a href="#sc_WatchGuarantees">What ZooKeeper Guarantees about Watches</a>
@@ -202,10 +207,26 @@
</ul>
</li>
<li>
-<a href="#Consistency+Guarantees">Consistency Guarantees</a>
+<a href="#sc_ZooKeeperAccessControl">ZooKeeper access control using ACLs</a>
+<ul class="minitoc">
+<li>
+<a href="#sc_ACLPermissions">ACL Permissions</a>
+<ul class="minitoc">
+<li>
+<a href="#sc_BuiltinACLSchemes">Builtin ACL Schemes</a>
</li>
<li>
-<a href="#Bindings">Bindings</a>
+<a href="#Zookeeper+C+client+API">Zookeeper C client API</a>
+</li>
+</ul>
+</li>
+</ul>
+</li>
+<li>
+<a href="#ch_zkGuarantees">Consistency Guarantees</a>
+</li>
+<li>
+<a href="#ch_bindings">Bindings</a>
<ul class="minitoc">
<li>
<a href="#Java+Binding">Java Binding</a>
@@ -224,44 +245,35 @@
</ul>
</li>
<li>
-<a href="#Building+Blocks%3A+A+Guide+to+ZooKeeper+Operations">Building Blocks: A Guide to ZooKeeper Operations</a>
+<a href="#ch_guideToZkOperations">Building Blocks: A Guide to ZooKeeper Operations</a>
</li>
<li>
-<a href="#Program+Structure%2C+with+Simple+Example">Program Structure, with Simple Example</a>
+<a href="#ch_programStructureWithExample">Program Structure, with Simple Example</a>
</li>
<li>
-<a href="#Gotchas%3A+Common+Problems+and+Troubleshooting">Gotchas: Common Problems and Troubleshooting</a>
+<a href="#ch_gotchas">Gotchas: Common Problems and Troubleshooting</a>
</li>
</ul>
</div>
-<title>ZooKeeper Programmer's Guide</title>
-<subtitle>Developing Distributed Applications that use ZooKeeper</subtitle>
-<a name="_introduction"></a>
-<preface id="_introduction">
-
-<title>Introduction</title>
-
-
+<a name="N1000B"></a><a name="_introduction"></a>
+<h2 class="h3">Introduction</h2>
+<div class="section">
<p>This document is a guide for developers wishing to create
distributed applications that take advantage of ZooKeeper's coordination
services. It contains conceptual and practical information.</p>
-
-
-<p>The first four chapters of this guide present higher level
+<p>The first four sections of this guide present higher level
discussions of various ZooKeeper concepts. These are necessary both for an
understanding of how Zookeeper works as well how to work with it. It does
not contain source code, but it does assume a familiarity with the
- problems associated with distributed computing. The chapters in this first
+ problems associated with distributed computing. The sections in this first
group are:</p>
-
-
<ul>
<li>
@@ -300,12 +312,8 @@
</li>
</ul>
-
-
-<p>The next four chapters of this provided practical programming
+<p>The next four sections of this provided practical programming
information. These are:</p>
-
-
<ul>
<li>
@@ -330,8 +338,7 @@
<p>
<a href="#ch_programStructureWithExample">Program Structure, with Simple Example</a>
-
-<remark>[tbd]</remark>
+ <em>[tbd]</em>
</p>
</li>
@@ -346,25 +353,19 @@
</li>
</ul>
-
-
<p>The book concludes with an <a href="#apx_linksToOtherInfo">appendix</a> containing links to other
useful, ZooKeeper-related information.</p>
-
-
<p>Most of information in this document is written to be accessible as
stand-alone reference material. However, before starting your first
ZooKeeper application, you should probably at least read the chaptes on
the <a href="#ch_zkDataModel">ZooKeeper Data Model</a> and <a href="#ch_guideToZkOperations">ZooKeeper Basic Operations</a>. Also,
the <a href="#ch_programStructureWithExample">Simple Programmming
- Example</a>
-<remark>[tbd]</remark> is helpful for understand the basic
+ Example</a> <em>[tbd]</em> is helpful for understand the basic
structure of a ZooKeeper client application.</p>
-
-</preface>
+</div>
-<a name="N1007F"></a><a name="The+ZooKeeper+Data+Model"></a>
+<a name="N1007D"></a><a name="ch_zkDataModel"></a>
<h2 class="h3">The ZooKeeper Data Model</h2>
<div class="section">
<p>ZooKeeper has a hierarchal name space, much like a distributed file
@@ -418,7 +419,7 @@
</li>
</ul>
-<a name="N100A9"></a><a name="sc_zkDataModel_znodes"></a>
+<a name="N100A7"></a><a name="sc_zkDataModel_znodes"></a>
<h3 class="h4">ZNodes</h3>
<p>Every node in a ZooKeeper tree is refered to as a
<em>znode</em>. Znodes maintain a stat structure that
@@ -431,7 +432,7 @@
it must supply the version of the data of the znode it is changing. If
the version it supplies doesn't match the actual version of the data,
the update will fail. (This behavior can be overridden. For more
- information see... )<remark>[tbd...]</remark>
+ information see... )<em>[tbd...]</em>
</p>
<div class="note">
<div class="label">Note</div>
@@ -450,34 +451,34 @@
</div>
<p>Znodes are the main enitity that a programmer access. They have
several characteristics that are worth mentioning here.</p>
-<a name="N100CC"></a><a name="sc_zkDataMode_watches"></a>
+<a name="N100CA"></a><a name="sc_zkDataMode_watches"></a>
<h4>Watches</h4>
<p>Clients can set watches on znodes. Changes to that znode trigger
the watch and then clear the watch. When a watch triggers, ZooKeeper
sends the client a notification. More information about watches can be
found in the section
<a href="#ch_zkWatches">Zookeeper Watches</a>.
- <remark>[tbd]</remark>
+ <em>[tbd]</em>
</p>
-<a name="N100DC"></a><a name="Data+Access"></a>
+<a name="N100DA"></a><a name="Data+Access"></a>
<h4>Data Access</h4>
<p>The data stored at each znode in a namespace is read and written
atomically. Reads get all the data bytes associated with a znode and a
write replaces all the data. Each node has an Access Control List
(ACL) that restricts who can do what.</p>
-<a name="N100E6"></a><a name="Ephemeral+Nodes"></a>
+<a name="N100E4"></a><a name="Ephemeral+Nodes"></a>
<h4>Ephemeral Nodes</h4>
<p>ZooKeeper also has the notion of ephemeral nodes. These znodes
exists as long as the session that created the znode is active. When
the session ends the znode is deleted. Because of this behavior
ephemeral znodes are not allowed to have children.</p>
-<a name="N100F0"></a><a name="Unique+Naming"></a>
+<a name="N100EE"></a><a name="Unique+Naming"></a>
<h4>Unique Naming</h4>
<p>Finally you create a znode, you can request that ZooKeeper
append a monotonicly increasing counter be appended to the path name
of the znode to be requested. This counter is unique to the parent
znode.</p>
-<a name="N100FB"></a><a name="sc_timeInZk"></a>
+<a name="N100F9"></a><a name="sc_timeInZk"></a>
<h3 class="h4">Time in ZooKeeper</h3>
<p>ZooKeeper tracks time multiple ways:</p>
<ul>
@@ -546,7 +547,7 @@
</li>
</ul>
-<a name="N10133"></a><a name="sc_zkStatStructure"></a>
+<a name="N10131"></a><a name="sc_zkStatStructure"></a>
<h3 class="h4">ZooKeeper Stat Structure</h3>
<p>The Stat structure for each znode in ZooKeeper is made up of the
following fields:</p>
@@ -656,7 +657,7 @@
</div>
-<a name="N10191"></a><a name="ZooKeeper+Sessions"></a>
+<a name="N1018F"></a><a name="ch_zkSessions"></a>
<h2 class="h3">ZooKeeper Sessions</h2>
<div class="section">
<p>When a client gets a handle to the ZooKeeper service, ZooKeeper
@@ -684,7 +685,7 @@
</div>
-<a name="N101A1"></a><a name="ZooKeeper+Watches"></a>
+<a name="N1019F"></a><a name="ch_zkWatches"></a>
<h2 class="h3">ZooKeeper Watches</h2>
<div class="section">
<p>All of the read operations in ZooKeeper - <strong>getData()</strong>, <strong>getChildren()</strong>, and <strong>exists()</strong> - have the option of setting a watch as a
@@ -759,7 +760,7 @@
client gets a disconnect event, it must consider that an implicit trigger
of all watches. When a client reconnects to a new server, the client
should re-set any watches that it is still interested in.</p>
-<a name="N101D7"></a><a name="sc_WatchGuarantees"></a>
+<a name="N101D5"></a><a name="sc_WatchGuarantees"></a>
<h3 class="h4">What ZooKeeper Guarantees about Watches</h3>
<p>With regard to watches, ZooKeeper maintains these
guarantees:</p>
@@ -794,7 +795,7 @@
</li>
</ul>
-<a name="N101FC"></a><a name="sc_WatchRememberThese"></a>
+<a name="N101FA"></a><a name="sc_WatchRememberThese"></a>
<h3 class="h4">Things to Remember about Watches</h3>
<ul>
@@ -837,7 +838,267 @@
</div>
-<a name="N1021F"></a><a name="Consistency+Guarantees"></a>
+<a name="N1021D"></a><a name="sc_ZooKeeperAccessControl"></a>
+<h2 class="h3">ZooKeeper access control using ACLs</h2>
+<div class="section">
+<p>ZooKeeper uses ACLs to control access to its znodes (the data nodes of a ZooKeeper data tree). The ACL implementation is quite similar to UNIX file access permissions: it employs permission bits to allow/disallow various operations against a node and the scope to which the bits apply. Unlike standard UNIX permissions, a ZooKeeper node is not limited by the three standard scopes for user (owner of the file), group, and world (other). ZooKeeper does not have a notion of an owner of a znode. Instead, an ACL specifies sets of ids and permissions that are associated with those ids.</p>
+<p>ZooKeeper supports pluggable authentication schemes. Ids are specified using the form <em>scheme:id</em>, where <em>scheme</em> is a the authentication scheme that the id corresponds to. For example, <em>host:host1.corp.com</em> is an id for a host named <em>host1.corp.com</em>.</p>
+<p>When a client connects to ZooKeeper and authenticates itself, ZooKeeper associates all the ids that correspond to a client with the clients connection. These ids are checked against the ACLs of znodes when a clients tries to access a node. ACLs are made up of pairs of <em>(scheme:expression, perms)</em>. The format of the <em>expression</em> is specific to the scheme. For example, the pair <em>(ip:19.22.0.0/16, READ)</em> gives the <em>READ</em> permission to any clients with an IP address that starts with 19.22.</p>
+<a name="N10244"></a><a name="sc_ACLPermissions"></a>
+<h3 class="h4">ACL Permissions</h3>
+<p>Zookeeper supports the following permissions:</p>
+<ul>
+
+<li>
+<p>
+<strong>CREATE</strong>: you can create a child node</p>
+</li>
+
+<li>
+<p>
+<strong>READ</strong>: you can get data from a node and list its children.</p>
+</li>
+
+<li>
+<p>
+<strong>WRITE</strong>: you can set data for a node</p>
+</li>
+
+<li>
+<p>
+<strong>DELETE</strong>: you can delete a child node</p>
+</li>
+
+<li>
+<p>
+<strong>ADMIN</strong>: you can set permissions</p>
+</li>
+
+</ul>
+<p>The <em>CREATE</em> and <em>DELETE</em> permissions have been broken out of the <em>WRITE</em> permission for finer grained access controls. The cases for <em>CREATE</em> and <em>DELETE</em> are the following:</p>
+<p>You want A to be able to do a set on a zookeeper node, but not be able to <em>CREATE</em> or <em>DELETE</em> children.</p>
+<p>
+<em>CREATE</em> without <em>DELETE</em>: clients create requests by creating zookeeper nodes in a parent directory. You want all clients to be able to add, but only request processor can delete. (This is kind of like the APPEND permission for files.)</p>
+<p>Also, the <em>ADMIN</em> permission is there since Zookeeper doesn’t have a notion of file owner. In some sense the <em>ADMIN</em> permission designates the entity as the owner. Zookeeper doesn’t support the LOOKUP permission (execute permission bit on directories to allow you to LOOKUP even though you can't list the directory). Everyone implicitly has LOOKUP permission. This allows you to stat a node, but nothing more. (The problem is, if you want to call zoo_exists() on a node that doesn't exist, there is no permission to check.)</p>
+<a name="N1029A"></a><a name="sc_BuiltinACLSchemes"></a>
+<h4>Builtin ACL Schemes</h4>
+<p>ZooKeeeper has the following built in schemes:</p>
+<ul>
+
+<li>
+<p>
+<strong>world</strong> has a single id, <em>anyone</em>, that represents anyone.</p>
+</li>
+
+<li>
+<p>
+<strong>auth</strong> doesn't use any id, represents any authenticated user.</p>
+</li>
+
+<li>
+<p>
+<strong>digest</strong> uses a <em>username:password</em> string to generate MD5 hash which is then used as an ACL ID identity. Authentication is done by sending the <em>username:password</em> in clear text. When used in the ACL the expression will be the <em>username:base64</em>encoded<em>SHA1</em>password<em>digest</em>.</p>
+</li>
+
+<li>
+<p>
+<strong>host</strong> uses the client host name as an ACL ID identity. The ACL expression is a hostname suffix. For example, the ACL expression <em>host:corp.com</em> matches the ids <em>host:host1.corp.com</em> and <em>host:host2.corp.com</em>, but not <em>host:host1.store.com</em>.</p>
+</li>
+
+<li>
+<p>
+<strong>ip</strong> uses the client host IP as an ACL ID identity. The ACL expression is of the form <em>addr/bits</em> where the most significant <em>bits</em> of <em>addr</em> are matched against the most significant <em>bits</em> of the client host IP.</p>
+</li>
+
+</ul>
+<a name="N102EF"></a><a name="Zookeeper+C+client+API"></a>
+<h4>Zookeeper C client API</h4>
+<p>The following constants are provided by the zookeeper C library:</p>
+<ul>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_READ; //can read node’s value and list its children</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_WRITE;// can set the node’s value</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_CREATE; //can create children</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_DELETE;// can delete children</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_ADMIN; //can execute set_acl()</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>int</em> PERM_ALL;// all of the above flags OR’d together</p>
+</li>
+
+</ul>
+<p>The following are the standard ACL IDs:</p>
+<ul>
+
+<li>
+<p>
+<em>struct</em> Id ANYONE_ID_UNSAFE; //(‘world’,’anyone’)</p>
+</li>
+
+<li>
+<p>
+<em>struct</em> Id AUTH_IDS;// (‘auth’,’’)</p>
+</li>
+
+</ul>
+<p>AUTH_IDS empty identity string should be interpreted as “the identity of the creator”.</p>
+<p>Zookeeper client comes with three standard ACLs:</p>
+<ul>
+
+<li>
+<p>
+<em>struct</em> ACL_vector OPEN_ACL_UNSAFE; //(PERM_ALL,ANYONE_ID_UNSAFE)</p>
+</li>
+
+<li>
+<p>
+<em>struct</em> ACL_vector READ_ACL_UNSAFE;// (PERM_READ, ANYONE_ID_UNSAFE)</p>
+</li>
+
+<li>
+<p>
+<em>struct</em> ACL_vector CREATOR_ALL_ACL; //(PERM_ALL,AUTH_IDS)</p>
+</li>
+
+</ul>
+<p>The OPEN_ACL_UNSAFE is completely open free for all ACL: any application can execute any operation on the node and can create, list and delete its children. The READ_ACL_UNSAFE is read-only access for any application. CREATE_ALL_ACL grants all permissions to the creator of the node. The creator must have been authenticated by the server (for example, using “<em>digest</em>” scheme) before it can create nodes with this ACL.</p>
+<p>The following zookeeper operations deal with ACLs:</p>
+<ul>
+
+<li>
+<p>
+<em>int</em> <em>zoo_add_auth</em>(zhandle_t *zh,<em>const</em> <em>char</em>* scheme,<em>const</em> <em>char</em>* cert,</p>
+</li>
+
+<li>
+<p>
+<em>int</em> certLen, void_completion_t completion, <em>const</em> <em>void</em> *data);</p>
+</li>
+
+</ul>
+<p>The application uses the zoo_add_auth function to authenticate itself to the server. The function can be called multiple times if the application wants to authenticate using different schemes and/or identities.</p>
+<ul>
+
+<li>
+<p>
+<em>int</em> <em>zoo_create</em>(zhandle_t *zh, <em>const</em> <em>char</em> *path, <em>const</em> <em>char</em> *value,</p>
+</li>
+
+<li>
+<p>
+<em>int</em> valuelen, <em>const</em> <em>struct</em> ACL_vector *acl, <em>int</em> flags,</p>
+</li>
+
+<li>
+<p>
+<em>char</em> *realpath, <em>int</em> max_realpath_len);</p>
+</li>
+
+</ul>
+<p>zoo_create() operation creates a new node. The acl parameter is a list of ACLs associated with the node. The parent node must have the CREATE permission bit set.</p>
+<ul>
+
+<li>
+<p>
+<em>int</em> <em>zoo_get_acl</em>(zhandle_t *zh, <em>const</em> <em>char</em> *path,</p>
+</li>
+
+<li>
+<p>
+<em>struct</em> ACL_vector *acl, <em>struct</em> Stat *stat);</p>
+</li>
+
+</ul>
+<p>This operation returns a node’s ACL info.</p>
+<ul>
+
+<li>
+<p>
+<em>int</em> <em>zoo_set_acl</em>(zhandle_t *zh, <em>const</em> <em>char</em> *path, <em>int</em> version,</p>
+</li>
+
+<li>
+<p>
+<em>const</em> <em>struct</em> ACL_vector *acl);</p>
+</li>
+
+</ul>
+<p>This function replaces node’s ACL list with a new one. The node must have the ADMIN permission set.</p>
+<p>Here is a sample code that makes use of the above APIs to authenticate itself using the “<em>foo</em>” scheme and create an ephemeral node “/xyz” with create-only permissions.</p>
+<pre class="code">
+static zhandle_t *zh;
+
+void watcher(zhandle_t *zzh, int type, int state, const char *path) {
+}
+
+int main(int argc, char argv) {
+ char buffer[512];
+ char p[2048];
+ char *cert=0;
+ char appId[64];
+
+ strcpy(appId, "example.foo_test");
+ cert = foo_get_cert_once(appId);
+ if(cert!=0) {
+ fprintf(stderr,
+ "Certificate for appid [%s] is [%s]\n",appId,cert);
+ strncpy(p,cert, sizeof(p)-1);
+ free(cert);
+ } else {
+ fprintf(stderr, "Certificate for appid [%s] not found\n",appId);
+ strcpy(p, "dummy");
+ }
+
+ zoo_set_debug_level(LOG_LEVEL_DEBUG);
+
+ zh = zookeeper_init(“localhost:3181”, watcher,10000,0, 0, 0);
+ if (!zh) {
+ return errno;
+ }
+ if(zoo_add_auth(zh,"foo",p,strlen(p),0,0)!=ZOK)
+ return 2;
+
+ struct ACL_CREATE_ONLY_ACL[] = {{PERM_CREATE, AUTH_IDS}};
+ struct ACL_vector CREATE_ONLY = {1,_CREATE_ONLY_ACL};
+ int rc = zoo_create(zh,"/xyz","value", 5, &CREATE_ONLY, EPHEMERAL,
+ buffer, sizeof(buffer)-1);
+
+ this operation will fail with a ZNOAUTH error
+ int buflen= sizeof(buffer);
+ struct Stat stat;
+ rc = zoo_get(zh, "/xyz",0,buffer,&buflen,&stat);
+ if (rc) {
+ fprintf(stderr, "Error %d for %s\n", rc, line);
+ }
+
+ zookeeper_close(zh);
+ return 0;
+}
+ </pre>
+</div>
+
+
+<a name="N1040C"></a><a name="ch_zkGuarantees"></a>
<h2 class="h3">Consistency Guarantees</h2>
<div class="section">
<p>ZooKeeper is a high performance, scalable service. Both reads and
@@ -890,7 +1151,7 @@
timeouts, etc) the client will not know if the update has
applied or not. We take steps to minimize the failures, but the
only guarantee is only present with successful return codes.
- (This is called the _monotonicity condition_ in Paxos.)</p>
+ (This is called the <em>monotonicity condition</em> in Paxos.)</p>
</li>
@@ -923,7 +1184,6 @@
revocable locks solely at the ZooKeeper client (no additions needed to
ZooKeeper). See <a href="recipes.html">Recipes and Solutions</a>
for more details.</p>
-<p>
<div class="note">
<div class="label">Note</div>
<div class="content">
@@ -954,25 +1214,23 @@
primitives can be used to construct higher level functions that
provide complete client synchronization. (For more information,
see the <a href="recipes.html#sc_recipes_Locks">Locks</a>
-
-<remark>[tbd:...]</remark> in <a href="recipes.html">Zookeeper Recipes</a>.
- <remark>[tbd:..]</remark>).</p>
+ <em>[tbd:...]</em> in <a href="recipes.html">Zookeeper Recipes</a>.
+ <em>[tbd:..]</em>).</p>
</dd>
</dl>
</div>
</div>
-</p>
</div>
-<a name="N1028B"></a><a name="Bindings"></a>
+<a name="N1047A"></a><a name="ch_bindings"></a>
<h2 class="h3">Bindings</h2>
<div class="section">
<p>The ZooKeeper client libraries come in two languages: Java and C.
The following sections describe these.</p>
-<a name="N10294"></a><a name="Java+Binding"></a>
+<a name="N10483"></a><a name="Java+Binding"></a>
<h3 class="h4">Java Binding</h3>
<p>There are two packages that make up the ZooKeeper Java binding:
<strong>org.apache.zookeeper</strong> and <strong>org.apache.zookeeper.data</strong>. The rest of the
@@ -1039,7 +1297,7 @@
(SESSION_EXPIRED and AUTH_FAILED), the ZooKeeper object becomes invalid,
the two threads shut down, and any further ZooKeeper calls throw
errors.</p>
-<a name="N102DD"></a><a name="C+Binding"></a>
+<a name="N104CC"></a><a name="C+Binding"></a>
<h3 class="h4">C Binding</h3>
<p>The C binding has a single-threaded and multi-threaded library.
The multi-threaded library is easiest to use and is most similar to the
@@ -1056,7 +1314,7 @@
(i.e. FreeBSD 4.x). In all other cases, application developers should
link with zookeeper_mt, as it includes support for both Sync and Async
API.</p>
-<a name="N102EC"></a><a name="Installation"></a>
+<a name="N104DB"></a><a name="Installation"></a>
<h4>Installation</h4>
<p>If you're building the client from a check-out from the Apache
repository, follow the steps outlined below. If you're building from a
@@ -1187,7 +1445,7 @@
</li>
</ol>
-<a name="N10395"></a><a name="Using+the+Client"></a>
+<a name="N10584"></a><a name="Using+the+Client"></a>
<h4>Using the Client</h4>
<p>You can test your client by running a zookeeper server (see
instructions on the project wiki page on how to run it) and connecting
@@ -1196,7 +1454,9 @@
zookeeper_mt library) is shown in this example, but you could also use
cli_st (singlethreaded, built against zookeeper_st library):</p>
<p>
-<pre class="code">$ cli_mt zookeeper_host:9876</pre>This
+<span class="codefrag command">$ cli_mt zookeeper_host:9876</span>
+</p>
+<p>This
is a client application that gives you a shell for executing simple
zookeeper commands. Once succesully started and connected to the
server it displays a shell prompt. You can now enter zookeeper
@@ -1222,27 +1482,29 @@
<p>If you are building a multithreaded client, compile with
-DTHREADED compiler flag to enable the multi-threaded version of
the library, and then link against against the
- <span class="codefrag varname">zookeeper_mt</span> library. If you are building a
+ <em>zookeeper_mt</em> library. If you are building a
single-threaded client, do not compile with -DTHREADED, and be
- sure to link against the<span class="codefrag varname"> zookeeper_st
- </span>library.</p>
+ sure to link against the<em> zookeeper_st
+ </em>library.</p>
</li>
</ol>
-<p>Refer to <a href="#ch_programStructureWithExample">Program Structure, with Simple Example</a>for examples of usage in Java and C.
- <remark>[tbd]</remark>
+<p>Refer to <a href="#ch_programStructureWithExample">Program Structure, with Simple Example</a>
+ for examples of usage in Java and C.
+ <em>[tbd]</em>
+
</p>
</div>
-<a name="N103D2"></a><a name="Building+Blocks%3A+A+Guide+to+ZooKeeper+Operations"></a>
+<a name="N105C3"></a><a name="ch_guideToZkOperations"></a>
<h2 class="h3">Building Blocks: A Guide to ZooKeeper Operations</h2>
<div class="section">
<p>
-<remark>[tbd: This is a new section. The below
+<em>[tbd: This is a new section. The below
is just placeholder. Eventually, a subsection on each of those operations, with a little
- bit of illustrative code for each op.] </remark>
+ bit of illustrative code for each op.] </em>
</p>
<p>One of the design goals of ZooKeeper is provide a very simple
programming interface. As a result, it supports only these
@@ -1308,16 +1570,16 @@
</div>
-<a name="N10414"></a><a name="Program+Structure%2C+with+Simple+Example"></a>
+<a name="N10605"></a><a name="ch_programStructureWithExample"></a>
<h2 class="h3">Program Structure, with Simple Example</h2>
<div class="section">
<p>
-<remark>[tbd]</remark>
+<em>[tbd]</em>
</p>
</div>
-<a name="N1041F"></a><a name="Gotchas%3A+Common+Problems+and+Troubleshooting"></a>
+<a name="N10610"></a><a name="ch_gotchas"></a>
<h2 class="h3">Gotchas: Common Problems and Troubleshooting</h2>
<div class="section">
<p>So now you know ZooKeeper. It's fast, simple, your application
@@ -1418,7 +1680,7 @@
<dl>
<dt>
-<term>ZooKeeper Whitepaper <remark>[tbd: find url]</remark>
+<term>ZooKeeper Whitepaper <em>[tbd: find url]</em>
</term>
</dt>
<dd>
@@ -1428,7 +1690,7 @@
<dt>
-<term>API Reference <remark>[tbd: find url]</remark>
+<term>API Reference <em>[tbd: find url]</em>
</term>
</dt>
<dd>
@@ -1485,7 +1747,7 @@
<dt>
<term>
-<remark>[tbd]</remark>
+<em>[tbd]</em>
</term>
</dt>
<dd>
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperStarted.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperStarted.html?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperStarted.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperStarted.html Fri Oct 3 16:35:27 2008
@@ -5,7 +5,7 @@
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.8">
<meta name="Forrest-skin-name" content="pelt">
-<title></title>
+<title>ZooKeeper Getting Started Guide</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
@@ -157,10 +157,11 @@
<a class="dida" href="zookeeperStarted.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
+<h1>ZooKeeper Getting Started Guide</h1>
<div id="minitoc-area">
<ul class="minitoc">
<li>
-<a href="#Getting+Started%3A+Coordinating+Distributed+Applications+with%0A++++++ZooKeeper">Getting Started: Coordinating Distributed Applications with
+<a href="#ch_GettingStarted">Getting Started: Coordinating Distributed Applications with
ZooKeeper</a>
<ul class="minitoc">
<li>
@@ -183,12 +184,11 @@
</ul>
</div>
-<title>ZooKeeper Getting Started Guide</title>
-<a name="N1000A"></a><a name="Getting+Started%3A+Coordinating+Distributed+Applications+with%0A++++++ZooKeeper"></a>
+<a name="N10009"></a><a name="ch_GettingStarted"></a>
<h2 class="h3">Getting Started: Coordinating Distributed Applications with
ZooKeeper</h2>
<div class="section">
@@ -201,7 +201,7 @@
deployments, and optimizing the transaction log. However for the complete
instructions for commercial deployments, please refer to the <a href="zookeeperAdmin.html">Zookeeper
Administrator's Guide</a>.</p>
-<a name="N10017"></a><a name="sc_InstallingSingleMode"></a>
+<a name="N10016"></a><a name="sc_InstallingSingleMode"></a>
<h3 class="h4">Installing and Running ZooKeeper in Single Server Mode</h3>
<p>Setting up a ZooKeeper server in standalone mode is
straightforward. The server is contained in a single JAR file, so
@@ -216,23 +216,21 @@
</div>
</div>
<p>Once you have downloaded the ZooKeeper source, cd to the root of
- your ZooKeeper source, and run "ant jar". For example:<pre class="code">$ cd ~/dev/zookeeper
+ your ZooKeeper source, and run "ant jar". For example:<span class="codefrag computeroutput">$ cd ~/dev/zookeeper
-$ ~/dev/zookeeper/: ant jar</pre>
+$ ~/dev/zookeeper/: ant jar</span>
</p>
<p>This should generate a JAR file called zookeeper.jar. To start
- Zookeeper, compile and run zookeeper.jar. <remark>[tbd, some more
+ Zookeeper, compile and run zookeeper.jar. <em>[tbd, some more
instruction here. Perhaps a command line? Are these two steps or
- one?]</remark>
+ one?]</em>
</p>
<p>To start ZooKeeper you need a configuration file. Here is a sample
file:</p>
-<p>
<pre class="code">tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181
</pre>
-</p>
<p>This file can be called anything, but for the sake of this
discussion, call it <strong>zoo.cfg</strong>. Here are
the meanings for each of the fields:</p>
@@ -277,12 +275,12 @@
<p>Now that you created the configuration file, you can start
ZooKeeper:</p>
<p>
-<pre class="code">java -cp zookeeper-dev.jar:java/lib/log4j-1.2.15.jar:conf org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg</pre>
+<span class="codefrag computeroutput">java -cp zookeeper-dev.jar:java/lib/log4j-1.2.15.jar:conf org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg</span>
</p>
<p>ZooKeeper logs messages using log4j -- more detail available in
the <a href="zookeeperProgrammers.html#Logging">Logging</a>
- section of the Programmer's Guide.<remark>[tbd:
- real reference needed]</remark> You will see log messages coming to the
+ section of the Programmer's Guide.<em>[tbd:
+ real reference needed]</em> You will see log messages coming to the
console and/or a log file depending on the log4j configuration.</p>
<p>The steps outlined here run ZooKeeper in standalone mode. There is
no replication, so if Zookeeper process fails, the service will go down.
@@ -344,13 +342,13 @@
file. The file is similar to the one used in standalone mode, but with a
few differences. Here is an example:</p>
<p>
-<pre class="code">tickTime=2000
+<span class="codefrag computeroutput">tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888 server.2=zoo2:2888
-server.3=zoo3:2888 </pre>
+server.3=zoo3:2888 </span>
</p>
<p>The new entry, <strong>initLimit</strong> is
timeouts ZooKeeper uses to limit the length of time the Zookeeper
@@ -367,7 +365,7 @@
ASCII.</p>
<p>Finally, note the "2888" port numbers after each server name.
These are the "electionPort" numbers of the servers (as opposed to
- clientPorts), that is ports for <remark>[tbd]</remark>.</p>
+ clientPorts), that is ports for <em>[tbd]</em>.</p>
<div class="note">
<div class="label">Note</div>
<div class="content">
@@ -379,7 +377,7 @@
</div>
</div>
-<a name="N100EB"></a><a name="Other+Optimizations"></a>
+<a name="N100EC"></a><a name="Other+Optimizations"></a>
<h3 class="h4">Other Optimizations</h3>
<p>There are a couple of other configuration parameters that can
greatly increase performance:</p>
@@ -399,7 +397,7 @@
<li>
<p>
-<remark>[tbd: what is the other config param?]</remark>
+<em>[tbd: what is the other config param?]</em>
</p>
</li>
Modified: hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml?rev=701567&r1=701566&r2=701567&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml Fri Oct 3 16:35:27 2008
@@ -15,12 +15,12 @@
limitations under the License.
-->
-<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
-"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
-<book id="ar_Recipes">
+<!DOCTYPE article PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN"
+"http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd">
+<article id="ar_Recipes">
<title>ZooKeeper Recipes and Solutions</title>
- <bookinfo>
+ <articleinfo>
<legalnotice>
<para>Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. You may
@@ -41,9 +41,9 @@
<para>$Revision: 1.6 $ $Date: 2008/09/19 03:46:18 $</para>
</abstract>
- </bookinfo>
+ </articleinfo>
- <chapter id="ch_recipes">
+ <section id="ch_recipes">
<title>A Guide to Creating Higher-level Constructs with ZooKeeper</title>
<para>In this article, you'll find guidelines for using
@@ -91,11 +91,12 @@
<section id="sc_recipes_eventHandles">
<title>Barriers</title>
- <para>Distributed systems use <firstterm>barriers</firstterm> to block
- processing of a set of nodes until a condition is met at which time all
- the nodes are allowed to proceed. Barriers are implemented in ZooKeeper by
- designating a barrier node. The barrier is in place if the barrier node
- exists. Here's the pseudo code:</para>
+ <para>Distributed systems use <emphasis>barriers</emphasis>
+ to block processing of a set of nodes until a condition is met
+ at which time all the nodes are allowed to proceed. Barriers are
+ implemented in ZooKeeper by designating a barrier node. The
+ barrier is in place if the barrier node exists. Here's the
+ pseudo code:</para>
<orderedlist>
<listitem>
@@ -140,91 +141,93 @@
the computation. (The <emphasis>x</emphasis> here is up to you to
determine for your system.)</para>
- <para><informaltable colsep="0" frame="none" rowsep="0">
- <tgroup cols="2">
- <tbody>
- <row>
- <entry align="center"><emphasis
- role="bold">Enter</emphasis></entry>
-
- <entry align="center"><emphasis
- role="bold">Leave</emphasis></entry>
- </row>
-
- <row>
- <entry align="left"><orderedlist>
- <listitem>
- <para>Create a name <emphasis><emphasis>n</emphasis> =
- <emphasis>b</emphasis>+â/â+<emphasis>p</emphasis></emphasis></para>
- </listitem>
-
- <listitem>
- <para>Set watch: <emphasis
- role="bold">exists(<emphasis>b</emphasis> + ââ/readyââ,
- true)</emphasis></para>
- </listitem>
-
- <listitem>
- <para>Create child: <emphasis role="bold">create(
- <emphasis>n</emphasis>, EPHEMERAL)</emphasis></para>
- </listitem>
-
- <listitem>
- <para><emphasis role="bold">L = getChildren(b,
- false)</emphasis></para>
- </listitem>
-
- <listitem>
- <para>if fewer children in L than<emphasis>
- x</emphasis>, wait for watch event</para>
- </listitem>
-
- <listitem>
- <para>else <emphasis role="bold">create(b + ââ/readyââ,
- REGULAR)</emphasis></para>
- </listitem>
- </orderedlist></entry>
-
- <entry><orderedlist>
- <listitem>
- <para><emphasis role="bold">L = getChildren(b,
- false)</emphasis></para>
- </listitem>
-
- <listitem>
- <para>if no children, exit</para>
- </listitem>
+ <informaltable colsep="0" frame="none" rowsep="0">
+ <tgroup cols="2">
+ <tbody>
+ <row>
+ <entry align="center"><emphasis
+ role="bold">Enter</emphasis></entry>
- <listitem>
- <para>if <emphasis>p</emphasis> is only process node in
+ <entry align="center"><emphasis
+ role="bold">Leave</emphasis></entry>
+ </row>
+
+ <row>
+ <entry align="left"><orderedlist>
+ <listitem>
+ <para>Create a name <emphasis><emphasis>n</emphasis> =
+ <emphasis>b</emphasis>+â/â+<emphasis>p</emphasis></emphasis></para>
+ </listitem>
+
+ <listitem>
+ <para>Set watch: <emphasis
+ role="bold">exists(<emphasis>b</emphasis> + ââ/readyââ,
+ true)</emphasis></para>
+ </listitem>
+
+ <listitem>
+ <para>Create child: <emphasis role="bold">create(
+ <emphasis>n</emphasis>, EPHEMERAL)</emphasis></para>
+ </listitem>
+
+ <listitem>
+ <para><emphasis role="bold">L = getChildren(b,
+ false)</emphasis></para>
+ </listitem>
+
+ <listitem>
+ <para>if fewer children in L than<emphasis>
+ x</emphasis>, wait for watch event</para>
+ </listitem>
+
+ <listitem>
+ <para>else <emphasis role="bold">create(b + ââ/readyââ,
+ REGULAR)</emphasis></para>
+ </listitem>
+ </orderedlist></entry>
+
+ <entry><orderedlist>
+ <listitem>
+ <para><emphasis role="bold">L = getChildren(b,
+ false)</emphasis></para>
+ </listitem>
+
+ <listitem>
+ <para>if no children, exit</para>
+ </listitem>
+
+ <listitem>
+ <para>if <emphasis>p</emphasis> is only process node in
L, delete(n) and exit</para>
- </listitem>
+ </listitem>
- <listitem>
- <para>if <emphasis>p</emphasis> is the lowest process
+ <listitem>
+ <para>if <emphasis>p</emphasis> is the lowest process
node in L, wait on highest process node in P</para>
- </listitem>
+ </listitem>
- <listitem>
- <para>else <emphasis
- role="bold">delete(<emphasis>n</emphasis>) </emphasis>if
+ <listitem>
+ <para>else <emphasis
+ role="bold">delete(<emphasis>n</emphasis>) </emphasis>if
still exists and wait on lowest process node in L</para>
- </listitem>
+ </listitem>
- <listitem>
- <para>goto 1</para>
- </listitem>
- </orderedlist></entry>
- </row>
- </tbody>
- </tgroup>
- </informaltable>On entering, all processes watch on a ready node and
- create an ephemeral node as a child of the barrier node. Each process
- but the last enters the barrier and waits for the ready node to appear
- at line 5. The process that creates the xth node, the last process, will
- see x nodes in the list of children and create the ready node, waking up
- the other processes. Note that waiting processes wake up only when it is
- time to exit, so waiting is efficient.</para>
+ <listitem>
+ <para>goto 1</para>
+ </listitem>
+ </orderedlist></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+ <para>On entering, all processes watch on a ready node and
+ create an ephemeral node as a child of the barrier node. Each process
+ but the last enters the barrier and waits for the ready node to appear
+ at line 5. The process that creates the xth node, the last process, will
+ see x nodes in the list of children and create the ready node, waking up
+ the other processes. Note that waiting processes wake up only when it is
+ time to exit, so waiting is efficient.
+ </para>
<para>On exit, you can't use a flag such as <emphasis>ready</emphasis>
because you are watching for process nodes to go away. By using
@@ -370,7 +373,7 @@
<listitem>
<para>Call <emphasis role="bold">create( )</emphasis> to
create a node with pathname
- "<parameter>_locknode_/read-</parameter>". This is the
+ "<filename>_locknode_/read-</filename>". This is the
lock node use later in the protocol. Make sure to set both
the <emphasis>sequence</emphasis> and
<emphasis>ephemeral</emphasis> flags.</para>
@@ -385,7 +388,7 @@
<listitem>
<para>If there are no children with a pathname starting
- with "<parameter>write-</parameter>" and having a lower
+ with "<filename>write-</filename>" and having a lower
sequence number than the node created in step <emphasis
role="bold">1</emphasis>, the client has the lock and can
exit the protocol. </para>
@@ -395,7 +398,7 @@
<para>Otherwise, call <emphasis role="bold">exists(
)</emphasis>, with <emphasis>watch</emphasis> flag, set on
the node in lock directory with pathname staring with
- "<parameter>write-</parameter>" having the next lowest
+ "<filename>write-</filename>" having the next lowest
sequence number.</para>
</listitem>
@@ -416,7 +419,7 @@
<listitem>
<para>Call <emphasis role="bold">create( )</emphasis> to
create a node with pathname
- "<parameter>_locknode_/write-</parameter>". This is the
+ "<filename>_locknode_/write-</filename>". This is the
lock node spoken of later in the protocol. Make sure to
set both <emphasis>sequence</emphasis> and
<emphasis>ephemeral</emphasis> flags.</para>
@@ -457,16 +460,18 @@
</tgroup>
</informaltable>
- <para><note>
- <para>It might appear that this recipe creates a herd effect: when
- there is a large group of clients waiting for a read lock, and all
- getting notified more or less simultaneously when the
- "<parameter>write-</parameter>" node with the lowest sequence number
- is deleted. In fact. that's valid behavior: as all those waiting
- reader clients should be released since they have the lock. The herd
- effect refers to releasing a "herd" when in fact only a single or a
- small number of machines can proceed.</para>
- </note></para>
+ <note>
+ <para>It might appear that this recipe creates a herd effect:
+ when there is a large group of clients waiting for a read
+ lock, and all getting notified more or less simultaneously
+ when the "<filename>write-</filename>" node with the lowest
+ sequence number is deleted. In fact. that's valid behavior:
+ as all those waiting reader clients should be released since
+ they have the lock. The herd effect refers to releasing a
+ "herd" when in fact only a single or a small number of
+ machines can proceed.
+ </para>
+ </note>
</section>
<section id="sc_recoverableSharedLocks">
@@ -614,5 +619,5 @@
the current leader. Applications may consider creating a separate to znode
to acknowledge that the leader has executed the leader procedure. </para>
</section>
- </chapter>
-</book>
+ </section>
+</article>