You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by ph...@apache.org on 2008/10/02 00:28:36 UTC
svn commit: r700955 - in /hadoop/zookeeper/trunk: ./ docs/
src/docs/src/documentation/ src/docs/src/documentation/content/xdocs/
Author: phunt
Date: Wed Oct 1 15:28:35 2008
New Revision: 700955
URL: http://svn.apache.org/viewvc?rev=700955&view=rev
Log:
ZOOKEEPER-147. This is version of the documents with most of the [tbd...] scrubbed out
Added:
hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt (with props)
Modified:
hadoop/zookeeper/trunk/CHANGES.txt
hadoop/zookeeper/trunk/docs/recipes.html
hadoop/zookeeper/trunk/docs/recipes.pdf
hadoop/zookeeper/trunk/docs/zookeeperAdmin.html
hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf
hadoop/zookeeper/trunk/docs/zookeeperOver.html
hadoop/zookeeper/trunk/docs/zookeeperOver.pdf
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
hadoop/zookeeper/trunk/docs/zookeeperStarted.html
hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml
Modified: hadoop/zookeeper/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/CHANGES.txt?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/CHANGES.txt (original)
+++ hadoop/zookeeper/trunk/CHANGES.txt Wed Oct 1 15:28:35 2008
@@ -13,13 +13,13 @@
ZOOKEEPER-38. headers (version+) in log/snap files (Andrew Kornev and Mahadev
Konar via breed)
- ZOOKEEPER-127. Use of non-standard election ports in config breaks services
- (Mark Harwood and Flavio Junqueira via breed)
-
Backward compatibile changes:
BUGFIXES:
+ ZOOKEEPER-147. This is version of the documents with most of the [tbd...]
+ scrubbed out (robbie via phunt)
+
ZOOKEEPER-150. zookeeper build broken (mahadev via phunt)
ZOOKEEPER-144. add tostring support for watcher event, and enums for event
@@ -104,3 +104,7 @@
ZOOKEEPER-117. threading issues in Leader election (Flavio Junqueira and Patrick
Hunt via breed)
+
+ ZOOKEEPER-127. Use of non-standard election ports in config breaks services
+ (Mark Harwood and Flavio Junqueira via breed)
+
Modified: hadoop/zookeeper/trunk/docs/recipes.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/recipes.html?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/recipes.html (original)
+++ hadoop/zookeeper/trunk/docs/recipes.html Wed Oct 1 15:28:35 2008
@@ -288,11 +288,7 @@
</li>
</ol>
-<p>
-<remark>[tbd: maybe an illustration would be nice for each of the
- recipes?]</remark>
-</p>
-<a name="N1006C"></a><a name="sc_doubleBarriers"></a>
+<a name="N10068"></a><a name="sc_doubleBarriers"></a>
<h4>Double Barriers</h4>
<p>Double barriers enable clients to synchronize the beginning and
the end of a computation. When enough processes have joined the barrier,
@@ -364,9 +360,7 @@
<li>
<p>if fewer children in L than<em>
- x</em>, wait for watch event <remark>[tbd: how do
- you wait?]</remark>
-</p>
+ x</em>, wait for watch event</p>
</li>
@@ -459,7 +453,7 @@
(picking the highest for simplicity) to go away. This means that only a
single process wakes up on each node deletion except for the last node,
which wakes up everyone when it is removed.</p>
-<a name="N10120"></a><a name="sc_recipes_Queues"></a>
+<a name="N1011A"></a><a name="sc_recipes_Queues"></a>
<h3 class="h4">Queues</h3>
<p>Distributed queues are a common data structure. To implement a
distributed queue in ZooKeeper, first designate a znode to hold the queue,
@@ -476,7 +470,7 @@
the list obtained from the first <strong>getChildren(
)</strong> call. If there are are no children in the queue node, the
reader waits for a watch notification to check to queue again.</p>
-<a name="N1013E"></a><a name="sc_recipes_priorityQueues"></a>
+<a name="N10138"></a><a name="sc_recipes_priorityQueues"></a>
<h4>Priority Queues</h4>
<p>To implement a priority queue, you need only make two simple
changes to the generic <a href="#sc_recipes_Queues">queue
@@ -486,7 +480,7 @@
from the queue a client uses an up-to-date children list meaning that
the client will invalidate previously obtained children lists if a watch
notification triggers for the queue node.</p>
-<a name="N1014D"></a><a name="sc_recipes_Locks"></a>
+<a name="N10147"></a><a name="sc_recipes_Locks"></a>
<h3 class="h4">Locks</h3>
<p>Fully distributed locks that are globally synchronous, meaning at
any snapshot in time no two clients think they hold the same lock. These
@@ -574,7 +568,7 @@
</li>
</ul>
-<a name="N101B9"></a><a name="Shared+Locks"></a>
+<a name="N101B3"></a><a name="Shared+Locks"></a>
<h4>Shared Locks</h4>
<p>You can implement shared locks by with a few changes to the lock
protocol:</p>
@@ -731,14 +725,12 @@
is deleted. In fact. that's valid behavior: as all those waiting
reader clients should be released since they have the lock. The herd
effect refers to releasing a "herd" when in fact only a single or a
- small number of machines can proceed. <remark>[tbd: myabe helpful to
- indicate which step this refers to?]</remark>
-</p>
+ small number of machines can proceed.</p>
</div>
</div>
</p>
-<a name="N10288"></a><a name="sc_recoverableSharedLocks"></a>
+<a name="N10280"></a><a name="sc_recoverableSharedLocks"></a>
<h4>Recoverable Shared Locks</h4>
<p>With minor modifications to the Shared Lock protocol, you make
shared locks revocable by modifying the shared lock protocol:</p>
@@ -760,7 +752,7 @@
Laser Beams</em> by stipulating in your protocol that the revoker
is allowed to delete the lock node if after some length of time the lock
isn't deleted by the lock holder.</p>
-<a name="N102B4"></a><a name="sc_recipes_twoPhasedCommit"></a>
+<a name="N102AC"></a><a name="sc_recipes_twoPhasedCommit"></a>
<h3 class="h4">Two-phased Commit</h3>
<p>A two-phase commit protocol is an algorithm that lets all clients in
a distributed system agree either to commit a transaction or abort.</p>
@@ -792,7 +784,7 @@
<p>To address the second problem, you can have the coordinator
propagate the transaction to the sites, and have each site creating its
own ephemeral node.</p>
-<a name="N102CD"></a><a name="sc_leaderElection"></a>
+<a name="N102C5"></a><a name="sc_leaderElection"></a>
<h3 class="h4">Leader Election</h3>
<p>A simple way of doing leader election with ZooKeeper is to use the
<strong>SEQUENCE|EPHEMERAL</strong> flags when creating
Modified: hadoop/zookeeper/trunk/docs/recipes.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/recipes.pdf?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperAdmin.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperAdmin.html?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperAdmin.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperAdmin.html Wed Oct 1 15:28:35 2008
@@ -273,7 +273,7 @@
production environment.</p>
<a name="N10034"></a><a name="sc_systemReq"></a>
<h3 class="h4">System Requirements</h3>
-<p>Zookeeper runs in Java, release 1.6 or greater, as group of hosts
+<p>Zookeeper runs in Java, release 1.5 or greater, as group of hosts
called a quorum. Three Zookeeper hosts per quorum is the minimum
recommended quorum size. At Yahoo!, Zookeeper is usually deployed on
dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB
@@ -283,7 +283,7 @@
<p>For reliable ZooKeeper service, you should deploy ZooKeeper in a
cluster known as a <em>quorum</em>. As long as a majority
of the quorum are up, the service will be available. Because Zookeeper
- requires a majority <remark>[tbd: why?]</remark>, it is best to use an
+ requires a majority, it is best to use an
odd number of machines. For example, with four machines ZooKeeper can
only handle the failure of a single machine; if two machines fail, the
remaining two machines do not constitute a majority. However, with five
@@ -298,8 +298,7 @@
<p>Install the Java JDK:</p>
-<pre class="code">$yinst -i jdk-1.6.0.00_3 -br test <remark>[y! prop - replace with open equiv]</remark>
-</pre>
+<pre class="code">$yinst -i jdk-1.6.0.00_3 -br test </pre>
</li>
@@ -310,10 +309,7 @@
swapping, which will seriously degrade Zookeeper performance. To
determine the correct value, load tests, make sure you are well
below the usage limit that would cause you to swap. Be conservative
- - use a maximum heap size of 3GB for a 4GB machine. <remark>[tbd:
- where would they do this? Environment variable,
- etc?]</remark>
-</p>
+ - use a maximum heap size of 3GB for a 4GB machine.</p>
</li>
@@ -323,8 +319,7 @@
<p>Install the Zookeeper Server Package:</p>
-<pre class="code">$ yinst install -nostart zookeeper_server <remark>[Y! prop - replace with open eq]</remark>
-</pre>
+<pre class="code">$ yinst install -nostart zookeeper_server </pre>
</li>
@@ -388,9 +383,7 @@
<li>
<p>In Java, you can run the following command to execute
- simple operations:<remark> [tbd: also, maybe give some of those
- simple operations?]</remark>
-</p>
+ simple operations:</p>
<pre class="code">$ java -cp zookeeper.jar:java/lib/log4j-1.2.15.jar:conf \
@@ -421,8 +414,7 @@
<p>Running either program gives you a shell in which to execute
- simple file-system-like operations. <remark>[tbd: again, sample
- operations?]</remark> To connect to Zookeeper with the multithreaded
+ simple file-system-like operations. To connect to Zookeeper with the multithreaded
client, for example, you would run:</p>
@@ -431,7 +423,7 @@
</li>
</ol>
-<a name="N100CE"></a><a name="sc_singleAndDevSetup"></a>
+<a name="N100C0"></a><a name="sc_singleAndDevSetup"></a>
<h3 class="h4">Single Server and Developer Setup</h3>
<p>If you want to setup Zookeeper for development purposes, you will
probably want to setup a single server instance of Zookeeper, and then
@@ -450,7 +442,7 @@
</div>
-<a name="N100EF"></a><a name="Administration"></a>
+<a name="N100E1"></a><a name="Administration"></a>
<h2 class="h3">Administration</h2>
<div class="section">
<p>This chapter contains information about running and maintaining
@@ -502,17 +494,15 @@
</ul>
</p>
-<a name="N10122"></a><a name="sc_configuration"></a>
+<a name="N10114"></a><a name="sc_configuration"></a>
<h3 class="h4">Configuration Parameters</h3>
<p>ZooKeeper's behavior is governed by the ZooKeeper configuration
file. This file is designed so that the exact same file can be used by
all the servers that make up a ZooKeeper server assuming the disk
layouts are the same. If servers use different configuration files,
care must be taken to ensure that the list of servers in all of the
- different configuration files match.<remark> [tbd: reformat in
- standard form, with legal values, etc]</remark>
-</p>
-<a name="N1012D"></a><a name="sc_minimumConfiguration"></a>
+ different configuration files match.</p>
+<a name="N1011D"></a><a name="sc_minimumConfiguration"></a>
<h4>Minimum Configuration</h4>
<p>Here are the minimum configuration keywords that must be
defined in the configuration file:</p>
@@ -561,7 +551,7 @@
</dl>
-<a name="N10154"></a><a name="sc_advancedConfiguration"></a>
+<a name="N10144"></a><a name="sc_advancedConfiguration"></a>
<h4>Advanced Configuration</h4>
<p>The configuration settings in the section are optional. You
can use them to further fine tune the behaviour of your Zookeeper
@@ -650,13 +640,12 @@
performance. (Note: The system property has no zookeeper
prefix, and the configuration variable name is different from
the system property. Yes - it's not consistent, and it's
- annoying.<remark> [tbd: is there any explanation for
- this?]</remark>)</p>
+ annoying.)</p>
</dd>
</dl>
-<a name="N101B7"></a><a name="sc_clusterOptions"></a>
+<a name="N101A4"></a><a name="sc_clusterOptions"></a>
<h4>Cluster Options</h4>
<p>The options in this section are designed for use in quorums --
that is, when deploying clusters of servers.</p>
@@ -685,9 +674,7 @@
election algorithm is not "0". When the election algorithm is
"0" a UDP port with the same port number as the port listed in
the <strong>server.num</strong> option will be
- used. <remark>[tbd: should that be <strong>server.id</strong>? Also, why isn't server.id
- documented anywhere?]</remark>
-</p>
+ used.</p>
</dd>
@@ -713,9 +700,7 @@
the leader can be configured to not accept clients and focus
on coordination. The default to this option is yes, which
means that a leader will accept client connections.
- <remark>[tbd: how do you specifiy which server is the
- leader?]</remark>
-</p>
+ </p>
<div class="note">
<div class="label">Note</div>
<div class="content">
@@ -736,9 +721,7 @@
<p>(No Java system property)</p>
<p>servers making up the Zookeeper quorum. When the server
starts up, it determines which server it is by looking for the
- file <span class="codefrag filename">myid</span> in the data directory.<remark>
- [tdb: should we mention somewhere about creating this file,
- myid, in the setup procedure?]</remark> That file contains the
+ file <span class="codefrag filename">myid</span> in the data directory. That file contains the
server number, in ASCII, and it should match <strong>x</strong> in <strong>server.x</strong> in the left hand side of this
setting.</p>
<p>The list of servers that make up ZooKeeper servers that
@@ -747,9 +730,7 @@
<p>The port numbers <strong>nnnn</strong>
in this setting are the <em>electionPort</em>
numbers of the servers (as opposed to clientPorts).
- <remark>[tbd: is the next sentence explanation an of what the
- election port or is it a description of a special case?]
- </remark>If you want to test multiple servers on a single
+ If you want to test multiple servers on a single
machine, the individual choices of electionPort for each
server can be defined in each server's config files using the
line electionPort=xxxx to avoid clashes.</p>
@@ -763,15 +744,12 @@
<p>(No Java system property)</p>
<p>Amount of time, in ticks (see <a href="#id_tickTime">tickTime</a>), to allow followers to
sync with ZooKeeper. If followers fall too far behind a
- leader, they will be dropped. <remark>[tbd: is this a correct
- rewording: if followers fall beyond this limit, they are
- dropped?]</remark>
-</p>
+ leader, they will be dropped.</p>
</dd>
</dl>
<p></p>
-<a name="N10232"></a><a name="Unsafe+Options"></a>
+<a name="N10210"></a><a name="Unsafe+Options"></a>
<h4>Unsafe Options</h4>
<p>The following options can be useful, but be careful when you
use them. The risk of each is explained along with the explanation
@@ -787,9 +765,7 @@
<p>Requires updates to be synced to media of the
transaction log before finishing processing the update. If
this option is set to no, ZooKeeper will not require updates
- to be synced to the media. <remark>[tbd: useful because...,
- dangerous because...]</remark>
-</p>
+ to be synced to the media.</p>
</dd>
@@ -815,7 +791,7 @@
</dt>
<dd>
<p>(Java system property: <strong>zookeeper.skipACL</strong>)</p>
-<p>Skips ACL checks. <remark>[tbd: when? where?]</remark>
+<p>Skips ACL checks.
This results in a boost in throughput, but opens up full
access to the data tree to everyone.</p>
</dd>
@@ -823,7 +799,7 @@
</dl>
-<a name="N10269"></a><a name="sc_zkCommands"></a>
+<a name="N10242"></a><a name="sc_zkCommands"></a>
<h3 class="h4">Zookeeper Commands: The Four Letter Words</h3>
<p>Zookeeper responds to a small set of commands. Each command is composed of
four letters. You issue the commands to Zookeeper via telnet or nc, at
@@ -874,16 +850,10 @@
imok
</pre>
-<a name="N1029B"></a><a name="sc_monitoring"></a>
+<a name="N10274"></a><a name="sc_monitoring"></a>
<h3 class="h4">Monitoring</h3>
-<p>
-<remark>[tbd: Patrick, Ben, et al: I believe the Message Broker
- team does perform routine monitoring of Zookeeper. But I might be
- wrong. To your knowledge, is there any monitoring of a Zookeeper
- deployment that will a Zookeeper sys admin will want to do, outside of
- Yahoo?]</remark>
-</p>
-<a name="N102A6"></a><a name="sc_dataFileManagement"></a>
+<remark>[tbd]</remark>
+<a name="N1027E"></a><a name="sc_dataFileManagement"></a>
<h3 class="h4">Data File Management</h3>
<p>ZooKeeper stores its data in a data directory and its transaction
log in a transaction log directory. By default these two directories are
@@ -891,7 +861,7 @@
transaction log files in a separate directory than the data files.
Throughput increases and latency decreases when transaction logs reside
on a dedicated log devices.</p>
-<a name="N102AF"></a><a name="The+Data+Directory"></a>
+<a name="N10287"></a><a name="The+Data+Directory"></a>
<h4>The Data Directory</h4>
<p>This directory has two files in it:</p>
<ul>
@@ -937,14 +907,14 @@
idempotent nature of its updates. By replaying the transaction log
against fuzzy snapshots ZooKeeper gets the state of the system at the
end of the log.</p>
-<a name="N102EB"></a><a name="The+Log+Directory"></a>
+<a name="N102C3"></a><a name="The+Log+Directory"></a>
<h4>The Log Directory</h4>
<p>The Log Directory contains the ZooKeeper transaction logs.
Before any update takes place, ZooKeeper ensures that the transaction
that represents the update is written to non-volatile storage. A new
log file is started each time a snapshot is begun. The log file's
suffix is the first zxid written to that log.</p>
-<a name="N102F5"></a><a name="File+Management"></a>
+<a name="N102CD"></a><a name="File+Management"></a>
<h4>File Management</h4>
<p>The format of snapshot and log files does not change between
standalone ZooKeeper servers and different configurations of
@@ -961,7 +931,7 @@
needs the latest complete fuzzy snapshot and the log files from the
start of that snapshot. The PurgeTxnLog utility implements a simple
retention policy that administrators can use.</p>
-<a name="N10306"></a><a name="sc_commonProblems"></a>
+<a name="N102DE"></a><a name="sc_commonProblems"></a>
<h3 class="h4">Things to Avoid</h3>
<p>Here are some common problems you can avoid by configuring
ZooKeeper correctly:</p>
@@ -977,9 +947,7 @@
but things will really act strange if clients have a list of
ZooKeeper servers that are in different ZooKeeper clusters. Also,
the server lists in each Zookeeper server configuration file
- should be consistent with one another. <remark>[tbd: I'm assuming
- this last part is true. Is it?]</remark>
-</p>
+ should be consistent with one another.</p>
</dd>
@@ -1017,12 +985,10 @@
</dd>
</dl>
-<a name="N1032C"></a><a name="sc_bestPractices"></a>
+<a name="N10302"></a><a name="sc_bestPractices"></a>
<h3 class="h4">Best Practices</h3>
<p>For best results, take note of the following list of good
- Zookeeper practices. <remark>[tbd: I just threw this section in. Do we
- have list that is is different from the "things to avoid"? If not, I can
- easily remove this section.]</remark>
+ Zookeeper practices. <remark>[tbd...]</remark>
</p>
</div>
Modified: hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperAdmin.pdf?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperOver.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOver.html?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperOver.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperOver.html Wed Oct 1 15:28:35 2008
@@ -447,12 +447,9 @@
<p>
<a href="#fg_zkComponents">ZooKeeper Components</a> shows the high-level components
of the ZooKeeper service. With the exception of the request processor,
- <remark>[tbd: where does the request processor live?]</remark> each of
+ each of
the servers that make up the ZooKeeper service replicates its own copy
- of each of components. <remark>[tbd: I changed the wording in this
- sentence from the white paper. Can someone please make sure it is still
- correct?]</remark>
-</p>
+ of each of components.</p>
<p>
<table class="ForrestTable" cellspacing="1" cellpadding="4">
<tr>
@@ -489,7 +486,7 @@
calculates what the state of the system is when the write is to be
applied and transforms this into a transaction that captures this new
state.</p>
-<a name="N1012F"></a><a name="Uses"></a>
+<a name="N1012A"></a><a name="Uses"></a>
<h3 class="h4">Uses</h3>
<p>The programming interface to ZooKeeper is deliberately simple.
With it, however, you can implement higher order operations, such as
@@ -498,7 +495,7 @@
white paper and video presentation.]</remark> For more information, see
<remark>[tbd]</remark>
</p>
-<a name="N1013E"></a><a name="Performance"></a>
+<a name="N10139"></a><a name="Performance"></a>
<h3 class="h4">Performance</h3>
<p>ZooKeeper is designed to be highly performant. But is it? The
results of the ZooKeeper's development team at Yahoo! Research indicate
@@ -580,7 +577,7 @@
ZooKeeper takes less than 200ms to elect a new leader. Third, as
followers recover, ZooKeeper is able to raise throughput again once they
start processing requests.</p>
-<a name="N1018F"></a><a name="The+ZooKeeper+Project"></a>
+<a name="N1018A"></a><a name="The+ZooKeeper+Project"></a>
<h3 class="h4">The ZooKeeper Project</h3>
<p>ZooKeeper has been successfully used in industrial applications.
It is used at Yahoo! as the coordination and failure recovery service
Modified: hadoop/zookeeper/trunk/docs/zookeeperOver.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperOver.pdf?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperProgrammers.html Wed Oct 1 15:28:35 2008
@@ -395,8 +395,7 @@
<li>
-<p>The following characters are not allowed because <remark>[tbd:
- do we need reasons?]</remark> :\ud800 -uF8FFF, \uFFF0-uFFFF, \uXFFFE -
+<p>The following characters are not allowed: \ud800 -uF8FFF, \uFFF0-uFFFF, \uXFFFE -
\uXFFFF (where X is an digit 1 - E), \uF0000 - \uFFFFF.</p>
</li>
@@ -419,7 +418,7 @@
</li>
</ul>
-<a name="N100AC"></a><a name="sc_zkDataModel_znodes"></a>
+<a name="N100A9"></a><a name="sc_zkDataModel_znodes"></a>
<h3 class="h4">ZNodes</h3>
<p>Every node in a ZooKeeper tree is refered to as a
<em>znode</em>. Znodes maintain a stat structure that
@@ -432,8 +431,7 @@
it must supply the version of the data of the znode it is changing. If
the version it supplies doesn't match the actual version of the data,
the update will fail. (This behavior can be overridden. For more
- information see... <remark>[tbd... reference here to the section
- describing the special version number -1]</remark>
+ information see... )<remark>[tbd...]</remark>
</p>
<div class="note">
<div class="label">Note</div>
@@ -452,36 +450,34 @@
</div>
<p>Znodes are the main enitity that a programmer access. They have
several characteristics that are worth mentioning here.</p>
-<a name="N100CF"></a><a name="sc_zkDataMode_watches"></a>
+<a name="N100CC"></a><a name="sc_zkDataMode_watches"></a>
<h4>Watches</h4>
<p>Clients can set watches on znodes. Changes to that znode trigger
the watch and then clear the watch. When a watch triggers, ZooKeeper
sends the client a notification. More information about watches can be
found in the section
<a href="#ch_zkWatches">Zookeeper Watches</a>.
- <remark>[tbd: fix this link] [tbd: Ben there is note from to emphasize
- that "it is queued". What is "it" and is what we have here
- sufficient?]</remark>
+ <remark>[tbd]</remark>
</p>
-<a name="N100DF"></a><a name="Data+Access"></a>
+<a name="N100DC"></a><a name="Data+Access"></a>
<h4>Data Access</h4>
<p>The data stored at each znode in a namespace is read and written
atomically. Reads get all the data bytes associated with a znode and a
write replaces all the data. Each node has an Access Control List
(ACL) that restricts who can do what.</p>
-<a name="N100E9"></a><a name="Ephemeral+Nodes"></a>
+<a name="N100E6"></a><a name="Ephemeral+Nodes"></a>
<h4>Ephemeral Nodes</h4>
<p>ZooKeeper also has the notion of ephemeral nodes. These znodes
exists as long as the session that created the znode is active. When
the session ends the znode is deleted. Because of this behavior
ephemeral znodes are not allowed to have children.</p>
-<a name="N100F3"></a><a name="Unique+Naming"></a>
+<a name="N100F0"></a><a name="Unique+Naming"></a>
<h4>Unique Naming</h4>
<p>Finally you create a znode, you can request that ZooKeeper
append a monotonicly increasing counter be appended to the path name
of the znode to be requested. This counter is unique to the parent
znode.</p>
-<a name="N100FE"></a><a name="sc_timeInZk"></a>
+<a name="N100FB"></a><a name="sc_timeInZk"></a>
<h3 class="h4">Time in ZooKeeper</h3>
<p>ZooKeeper tracks time multiple ways:</p>
<ul>
@@ -550,7 +546,7 @@
</li>
</ul>
-<a name="N10136"></a><a name="sc_zkStatStructure"></a>
+<a name="N10133"></a><a name="sc_zkStatStructure"></a>
<h3 class="h4">ZooKeeper Stat Structure</h3>
<p>The Stat structure for each znode in ZooKeeper is made up of the
following fields:</p>
@@ -660,7 +656,7 @@
</div>
-<a name="N10194"></a><a name="ZooKeeper+Sessions"></a>
+<a name="N10191"></a><a name="ZooKeeper+Sessions"></a>
<h2 class="h3">ZooKeeper Sessions</h2>
<div class="section">
<p>When a client gets a handle to the ZooKeeper service, ZooKeeper
@@ -668,9 +664,7 @@
assigns to the client. If the client connects to a different ZooKeeper
server, it will send the session id as a part of the connection handshake.
As a security measure, the server creates a password for the session id
- that any ZooKeeper server can validate. <remark>[tbd: note from Ben:
- "perhaps capability is a better word." need clarification on that.]
- </remark>The password is sent to the client with the session id when the
+ that any ZooKeeper server can validate.The password is sent to the client with the session id when the
client establishes the session. The client sends this password with the
session id whenever it reestablishes the session with a new server.</p>
<p>One of the parameters to the ZooKeeper client library call to create
@@ -690,7 +684,7 @@
</div>
-<a name="N101A7"></a><a name="ZooKeeper+Watches"></a>
+<a name="N101A1"></a><a name="ZooKeeper+Watches"></a>
<h2 class="h3">ZooKeeper Watches</h2>
<div class="section">
<p>All of the read operations in ZooKeeper - <strong>getData()</strong>, <strong>getChildren()</strong>, and <strong>exists()</strong> - have the option of setting a watch as a
@@ -765,7 +759,7 @@
client gets a disconnect event, it must consider that an implicit trigger
of all watches. When a client reconnects to a new server, the client
should re-set any watches that it is still interested in.</p>
-<a name="N101DD"></a><a name="sc_WatchGuarantees"></a>
+<a name="N101D7"></a><a name="sc_WatchGuarantees"></a>
<h3 class="h4">What ZooKeeper Guarantees about Watches</h3>
<p>With regard to watches, ZooKeeper maintains these
guarantees:</p>
@@ -800,7 +794,7 @@
</li>
</ul>
-<a name="N10202"></a><a name="sc_WatchRememberThese"></a>
+<a name="N101FC"></a><a name="sc_WatchRememberThese"></a>
<h3 class="h4">Things to Remember about Watches</h3>
<ul>
@@ -843,7 +837,7 @@
</div>
-<a name="N10225"></a><a name="Consistency+Guarantees"></a>
+<a name="N1021F"></a><a name="Consistency+Guarantees"></a>
<h2 class="h3">Consistency Guarantees</h2>
<div class="section">
<p>ZooKeeper is a high performance, scalable service. Both reads and
@@ -961,8 +955,8 @@
provide complete client synchronization. (For more information,
see the <a href="recipes.html#sc_recipes_Locks">Locks</a>
-<remark>[tbd: fix final link target]</remark> in <a href="recipes.html">Zookeeper Recipes</a>.
- <remark>[tbd: fix final link target]</remark>).</p>
+<remark>[tbd:...]</remark> in <a href="recipes.html">Zookeeper Recipes</a>.
+ <remark>[tbd:..]</remark>).</p>
</dd>
</dl>
@@ -973,12 +967,12 @@
</div>
-<a name="N10291"></a><a name="Bindings"></a>
+<a name="N1028B"></a><a name="Bindings"></a>
<h2 class="h3">Bindings</h2>
<div class="section">
<p>The ZooKeeper client libraries come in two languages: Java and C.
The following sections describe these.</p>
-<a name="N1029A"></a><a name="Java+Binding"></a>
+<a name="N10294"></a><a name="Java+Binding"></a>
<h3 class="h4">Java Binding</h3>
<p>There are two packages that make up the ZooKeeper Java binding:
<strong>org.apache.zookeeper</strong> and <strong>org.apache.zookeeper.data</strong>. The rest of the
@@ -1045,7 +1039,7 @@
(SESSION_EXPIRED and AUTH_FAILED), the ZooKeeper object becomes invalid,
the two threads shut down, and any further ZooKeeper calls throw
errors.</p>
-<a name="N102E3"></a><a name="C+Binding"></a>
+<a name="N102DD"></a><a name="C+Binding"></a>
<h3 class="h4">C Binding</h3>
<p>The C binding has a single-threaded and multi-threaded library.
The multi-threaded library is easiest to use and is most similar to the
@@ -1062,7 +1056,7 @@
(i.e. FreeBSD 4.x). In all other cases, application developers should
link with zookeeper_mt, as it includes support for both Sync and Async
API.</p>
-<a name="N102F2"></a><a name="Installation"></a>
+<a name="N102EC"></a><a name="Installation"></a>
<h4>Installation</h4>
<p>If you're building the client from a check-out from the Apache
repository, follow the steps outlined below. If you're building from a
@@ -1165,9 +1159,7 @@
<div class="content">
<p>See INSTALL for general information about running
- <strong>configure</strong>. <remark>[tbd: what
- is INSTALL? a directory? a file?]</remark>
-</p>
+ <strong>configure</strong>.</p>
</div>
</div>
@@ -1195,7 +1187,7 @@
</li>
</ol>
-<a name="N1039D"></a><a name="Using+the+Client"></a>
+<a name="N10395"></a><a name="Using+the+Client"></a>
<h4>Using the Client</h4>
<p>You can test your client by running a zookeeper server (see
instructions on the project wiki page on how to run it) and connecting
@@ -1212,9 +1204,7 @@
<pre class="code">> create /my_new_node</pre>
<p>To verify that the node's been created:</p>
<p>You should see a list of node who are children of the root node
- "/". <remark>[tbd: document all the cli commands (I think this is
- Ben's tbd? It's from sourceforge)]</remark>
-</p>
+ "/".</p>
<p>In order to be able to use the ZooKeeper API in your application
you have to remember to</p>
<ol>
@@ -1241,19 +1231,17 @@
</ol>
<p>Refer to <a href="#ch_programStructureWithExample">Program Structure, with Simple Example</a>for examples of usage in Java and C.
- <remark>[tbd: some kind of short tutorial would be helpful, I guess
- (ben's tbd?) ][tbd: whatever the case, make sure that link points to something.]</remark>
+ <remark>[tbd]</remark>
</p>
</div>
-<a name="N103DC"></a><a name="Building+Blocks%3A+A+Guide+to+ZooKeeper+Operations"></a>
+<a name="N103D2"></a><a name="Building+Blocks%3A+A+Guide+to+ZooKeeper+Operations"></a>
<h2 class="h3">Building Blocks: A Guide to ZooKeeper Operations</h2>
<div class="section">
<p>
-<remark>[Engineering input needed. This is a new section. The below
- is just placeholder, and was actually copied from the overview book. There
- should probably be a subsection on each of those operations, with a little
+<remark>[tbd: This is a new section. The below
+ is just placeholder. Eventually, a subsection on each of those operations, with a little
bit of illustrative code for each op.] </remark>
</p>
<p>One of the design goals of ZooKeeper is provide a very simple
@@ -1320,7 +1308,7 @@
</div>
-<a name="N1041E"></a><a name="Program+Structure%2C+with+Simple+Example"></a>
+<a name="N10414"></a><a name="Program+Structure%2C+with+Simple+Example"></a>
<h2 class="h3">Program Structure, with Simple Example</h2>
<div class="section">
<p>
@@ -1329,7 +1317,7 @@
</div>
-<a name="N10429"></a><a name="Gotchas%3A+Common+Problems+and+Troubleshooting"></a>
+<a name="N1041F"></a><a name="Gotchas%3A+Common+Problems+and+Troubleshooting"></a>
<h2 class="h3">Gotchas: Common Problems and Troubleshooting</h2>
<div class="section">
<p>So now you know ZooKeeper. It's fast, simple, your application
@@ -1485,8 +1473,7 @@
<dt>
<term>
-<a href="recipes.html">Zookeeper Recipes [tbd: fix
- linkend for apache site]</a>
+<a href="recipes.html">Zookeeper Recipes</a>
</term>
</dt>
<dd>
@@ -1502,7 +1489,7 @@
</term>
</dt>
<dd>
-<p>Whatever good sources anyone can think of...</p>
+<p>Any other good sources anyone can think of...</p>
</dd>
</dl>
Modified: hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperProgrammers.pdf?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
Binary files - no diff available.
Modified: hadoop/zookeeper/trunk/docs/zookeeperStarted.html
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperStarted.html?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/docs/zookeeperStarted.html (original)
+++ hadoop/zookeeper/trunk/docs/zookeeperStarted.html Wed Oct 1 15:28:35 2008
@@ -215,19 +215,15 @@
</div>
</div>
-<p>[tbd: should we start w/ a word here about were to get the source,
- exactly what to download, how to unpack it, and where to put it? Also,
- does the user need to be in sudo, or can they be under their regular
- login?]</p>
<p>Once you have downloaded the ZooKeeper source, cd to the root of
your ZooKeeper source, and run "ant jar". For example:<pre class="code">$ cd ~/dev/zookeeper
$ ~/dev/zookeeper/: ant jar</pre>
</p>
<p>This should generate a JAR file called zookeeper.jar. To start
- Zookeeper, compile and run zookeeper.jar. <em>[tbd, some more
+ Zookeeper, compile and run zookeeper.jar. <remark>[tbd, some more
instruction here. Perhaps a command line? Are these two steps or
- one?]</em>
+ one?]</remark>
</p>
<p>To start ZooKeeper you need a configuration file. Here is a sample
file:</p>
@@ -285,7 +281,7 @@
</p>
<p>ZooKeeper logs messages using log4j -- more detail available in
the <a href="zookeeperProgrammers.html#Logging">Logging</a>
- section of the Programmer's Guide.<remark revision="include_tbd">[tbd:
+ section of the Programmer's Guide.<remark>[tbd:
real reference needed]</remark> You will see log messages coming to the
console and/or a log file depending on the log4j configuration.</p>
<p>The steps outlined here run ZooKeeper in standalone mode. There is
@@ -294,7 +290,7 @@
replicated mode, please see <a href="#sc_RunningReplicatedZooKeeper">Running Replicated
Zookeeper</a>.</p>
<p></p>
-<a name="N1007A"></a><a name="sc_ConnectingToZooKeeper"></a>
+<a name="N10076"></a><a name="sc_ConnectingToZooKeeper"></a>
<h3 class="h4">Connecting to ZooKeeper</h3>
<p>Once ZooKeeper is running, you have several option for connection
to it:</p>
@@ -330,17 +326,15 @@
</li>
</ul>
-<a name="N100AB"></a><a name="sc_ProgrammingToZooKeeper"></a>
+<a name="N100A7"></a><a name="sc_ProgrammingToZooKeeper"></a>
<h3 class="h4">Programming to ZooKeeper</h3>
<p>ZooKeeper has a Java bindings and C bindings. They are
functionally equivalent. The C bindings exist in two variants: single
threaded and multi-threaded. These differ only in how the messaging loop
- is done. <remark>[tbd: what is the messaging loop? Do we talk about it
- anywyhere? is this too much info for a getting started guide?]</remark>
- For more information, see the <a href="zookeeperProgrammers.html#ch_programStructureWithExample.html">Programming
+ is done. For more information, see the <a href="zookeeperProgrammers.html#ch_programStructureWithExample.html">Programming
Examples in the Zookeeper Programmer's Guide</a> for
sample code using of the different APIs.</p>
-<a name="N100BC"></a><a name="sc_RunningReplicatedZooKeeper"></a>
+<a name="N100B5"></a><a name="sc_RunningReplicatedZooKeeper"></a>
<h3 class="h4">Running Replicated ZooKeeper</h3>
<p>Running ZooKeeper in standalone mode is convenient for evaluation,
some development, and testing. But in production, you should run
@@ -361,8 +355,7 @@
<p>The new entry, <strong>initLimit</strong> is
timeouts ZooKeeper uses to limit the length of time the Zookeeper
servers in quorum have to connect to a leader. The entry <strong>syncLimit</strong> limits how far out of date a server can
- be from a leader. [TBD: someone please verify that the previous is
- true.]</p>
+ be from a leader.</p>
<p>With both of these timeouts, you specify the unit of time using
<strong>tickTime</strong>. In this example, the timeout
for initLimit is 5 ticks at 2000 milleseconds a tick, or 10
@@ -374,8 +367,7 @@
ASCII.</p>
<p>Finally, note the "2888" port numbers after each server name.
These are the "electionPort" numbers of the servers (as opposed to
- clientPorts), that is ports for <remark>[tbd: feedback need: what are
- these ports, exactly?]</remark>.</p>
+ clientPorts), that is ports for <remark>[tbd]</remark>.</p>
<div class="note">
<div class="label">Note</div>
<div class="content">
@@ -387,7 +379,7 @@
</div>
</div>
-<a name="N100F2"></a><a name="Other+Optimizations"></a>
+<a name="N100EB"></a><a name="Other+Optimizations"></a>
<h3 class="h4">Other Optimizations</h3>
<p>There are a couple of other configuration parameters that can
greatly increase performance:</p>
@@ -407,8 +399,7 @@
<li>
<p>
-<remark>[tbd: feedback need: what is the other config param?
- (I believe two are mentioned above.)]</remark>
+<remark>[tbd: what is the other config param?]</remark>
</p>
</li>
Modified: hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/docs/zookeeperStarted.pdf?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
Binary files - no diff available.
Added: hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt?rev=700955&view=auto
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt (added)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt Wed Oct 1 15:28:35 2008
@@ -0,0 +1,227 @@
+This is a running list of todo documentation items. Feel free
+to add to the list or take on an item as you wish (in the form
+of a JIRA patch of course).
+-------------------------------------------------------------
+
+recipes.xml:110:
+[maybe an illustration would be nice for each recipe?]
+
+recipes.xml:167:
+"wait for each watch event". [how do you wait?]
+
+recipes.xml:457:
+<remark>[tbd: myabe helpful to indicate which step this refers to?]</remark>
+
+zookeeperAdmin.xml:77:
+because requires a majority <remark>[tbd: why?]</remark>, it is best to use...
+
+zookeeperAdmin.xml:112:
+ <screen>$yinst -i jdk-1.6.0.00_3 -br test <remark>[y! prop - replace with open equiv]</remark></screen>
+
+zookeeperAdmin.xml:99:
+- use a maximum heap size of 3GB for a 4GB machine. <remark>[tbd: where would they do this? Environment variable, etc?]</remark>
+
+zookeeperAdmin.xml:120
+<screen>$ yinst install -nostart zookeeper_server <remark>[Y! prop - replace with open eq]</remark></screen>
+
+zookeeperAdmin.xml:171:
+In Java, you can run the following command to execute simple operations:<remark> [tbd: also, maybe give some of those simple operations?]
+
+zookeeperAdmin.xml:194:
+Running either program gives you a shell in which to execute simple file-system-like operations. <remark>[tbd: again, sample
+ operations?]
+
+zookeeperAdmin.xml:252:
+If servers use different configuration files,
+care must be taken to ensure that the list of servers in all of the
+standard form, with legal values, etc]</remark>
+
+zookeeperAdmin.xml:408:
+(Note: The system property has no zookeeper
+prefix, and the configuration variable name is different from
+the system property. Yes - it's not consistent, and it's
+annoying.<remark> [tbd: is there any explanation for
+this?]</remark>)
+
+zookeeperAdmin.xml:445: When the election algorithm is
+ "0" a UDP port with the same port number as the port listed in
+ the <emphasis role="bold">server.num</emphasis> option will be
+ used. <remark>[tbd: should that be <emphasis
+ role="bold">server.id</emphasis>? Also, why isn't server.id
+ documented anywhere?]</remark>
+
+zookeeperAdmin.xml:481: The default to this option is yes, which
+ means that a leader will accept client connections.
+ <remark>[tbd: how do you specifiy which server is the
+ leader?]</remark>
+
+zookeeperAdmin.xml:495 When the server
+ starts up, it determines which server it is by looking for the
+ file <filename>myid</filename> in the data directory.<remark>
+ [tdb: should we mention somewhere about creating this file,
+ myid, in the setup procedure?]</remark>
+
+zookeeperAdmin.xml:508: [tbd: is the next sentence explanation an of what the
+ election port or is it a description of a special case?]
+ </remark>If you want to test multiple servers on a single
+ machine, the individual choices of electionPort for each
+ server can be defined in each server's config files using the
+ line electionPort=xxxx to avoid clashes.
+
+zookeeperAdmin.xml:524: If followers fall too far behind a
+ leader, they will be dropped. <remark>[tbd: is this a correct
+ rewording: if followers fall beyond this limit, they are
+ dropped?]</remark>
+
+zookeeperAdmin.xml:551: ZooKeeper will not require updates
+ to be synced to the media. <remark>[tbd: useful because...,
+ dangerous because...]</remark>
+
+zookeeperAdmin.xml:580: Skips ACL checks. <remark>[tbd: when? where?]</remark>
+
+zookeeperAdmin.xml:649: <remark>[tbd: Patrick, Ben, et al: I believe the Message Broker
+ team does perform routine monitoring of Zookeeper. But I might be
+ wrong. To your knowledge, is there any monitoring of a Zookeeper
+ deployment that will a Zookeeper sys admin will want to do, outside of
+ Yahoo?]</remark>
+
+zookeeperAdmin.xml:755: Also,
+ the server lists in each Zookeeper server configuration file
+ should be consistent with one another. <remark>[tbd: I'm assuming
+ this last part is true. Is it?]</remark>
+
+zookeeperAdmin.xml:812: For best results, take note of the following list of good
+ Zookeeper practices. <remark>[tbd: I just threw this section in. Do we
+ have list that is is different from the "things to avoid"? If not, I can
+ easily remove this section.]</remark>
+
+
+zookeeperOver.xml:162: Ephemeral nodes are useful when you
+ want to implement <remark>[tbd]</remark>.
+
+zookeeperOver.xml:174: And if the
+ connection between the client and one of the Zoo Keeper servers is
+ broken, the client will receive a local notification. These can be used
+ to <remark>[tbd]</remark>
+
+zookeeperOver.xml:215: <para>For more information on these (guarantees), and how they can be used, see
+ <remark>[tbd]</remark></para>
+
+zookeeperOver.xml:294: <para><xref linkend="fg_zkComponents" /> shows the high-level components
+ of the ZooKeeper service. With the exception of the request processor,
+ <remark>[tbd: where does the request processor live?]</remark>
+
+zookeeperOver.xml:298: <para><xref linkend="fg_zkComponents" /> shows the high-level components
+ of the ZooKeeper service. With the exception of the request processor,
+ each of
+ the servers that make up the ZooKeeper service replicates its own copy
+ of each of components. <remark>[tbd: I changed the wording in this
+ sentence from the white paper. Can someone please make sure it is still
+ correct?]</remark>
+
+zookeeperOver.xml:342: The programming interface to ZooKeeper is deliberately simple.
+ With it, however, you can implement higher order operations, such as
+ synchronizations primitives, group membership, ownership, etc. Some
+ distributed applications have used it to: <remark>[tbd: add uses from
+ white paper and video presentation.]</remark>
+
+
+zookeeperProgrammers.xml:94: <listitem>
+ <para><xref linkend="ch_programStructureWithExample" />
+ <remark>[tbd]</remark></para>
+ </listitem>
+
+zookeeperProgrammers.xml:115: Also,
+ the <ulink url="#ch_programStructureWithExample">Simple Programmming
+ Example</ulink> <remark>[tbd]</remark> is helpful for understand the basic
+ structure of a ZooKeeper client application.
+
+zookeeperProgrammers.xml:142: The following characters are not
+ allowed because <remark>[tbd:
+ do we need reasons?]</remark>
+
+zookeeperProgrammers.xml:172: If
+ the version it supplies doesn't match the actual version of the data,
+ the update will fail. (This behavior can be overridden. For more
+ information see... )<remark>[tbd... reference here to the section
+ describing the special version number -1]</remark>
+
+zookeeperProgrammers.xml:197: More information about watches can be
+ found in the section
+ <ulink url="recipes.html#sc_recipes_Locks">
+ Zookeeper Watches</ulink>.
+ <remark>[tbd: fix this link] [tbd: Ben there is note from to emphasize
+ that "it is queued". What is "it" and is what we have here
+ sufficient?]</remark></para>
+
+zookeeperProgrammers.xml:335: it will send the session id as a part of the connection handshake.
+ As a security measure, the server creates a password for the session id
+ that any ZooKeeper server can validate. <remark>[tbd: note from Ben:
+ "perhaps capability is a better word." need clarification on that.]
+ </remark>
+
+zookeeperProgrammers.xml:601: <ulink
+ url="recipes.html#sc_recipes_Locks">Locks</ulink>
+ <remark>[tbd:...]</remark> in <ulink
+ url="recipes.html">Zookeeper Recipes</ulink>.
+ <remark>[tbd:..]</remark>).</para>
+
+zookeeperProgrammers.xml:766: <para>See INSTALL for general information about running
+ <emphasis role="bold">configure</emphasis>. <remark>[tbd: what
+ is INSTALL? a directory? a file?]</remark></para>
+
+
+
+zookeeperProgrammers.xml:813: <para>To verify that the node's been created:</para>
+
+ <para>You should see a list of node who are children of the root node
+ "/".</para><remark>[tbd: document all the cli commands (I think this is ben's comment)
+
+zookeeperProgrammers.xml:838: <para>Refer to <xref linkend="ch_programStructureWithExample"/>for examples of usage in Java and C.
+ <remark>[tbd]</remark></para>
+
+zookeeperProgrammers.xml 847: <remark>[tbd: This is a new section. The below
+ is just placeholder. Eventually, a subsection on each of those operations, with a little
+ bit of illustrative code for each op.] </remark>
+
+zookeeperProgrammers.xml:915: Program Structure, with Simple Example</title>
+
+zookeeperProgrammers.xml:999: <term>ZooKeeper Whitepaper <remark>[tbd: find url]</remark></term>
+
+zookeeperProgrammers.xml:1008: <term>API Reference <remark>[tbd: find url]</remark></term>
+
+zookeeperProgrammers.xml:1062: [tbd]</remark></term><listitem>
+ <para>Any other good sources anyone can think of...</para>
+ </listitem>
+
+zookeeperStarted.xml:73: <para>[tbd: should we start w/ a word here about were to get the source,
+ exactly what to download, how to unpack it, and where to put it? Also,
+ does the user need to be in sudo, or can they be under their regular
+ login?]</para>
+
+zookeeperStarted.xml:84: <para>This should generate a JAR file called zookeeper.jar. To start
+ Zookeeper, compile and run zookeeper.jar. <emphasis>[tbd, some more
+ instruction here. Perhaps a command line? Are these two steps or
+ one?]</emphasis></para>
+
+zookeeperStarted.xml:139: <para>ZooKeeper logs messages using log4j -- more detail available in
+ the <ulink url="zookeeperProgrammers.html#Logging">Logging</ulink>
+ section of the Programmer's Guide.<remark revision="include_tbd">[tbd:
+ real reference needed]</remark>
+
+zookeeperStarted.xml:201: The C bindings exist in two variants: single
+ threaded and multi-threaded. These differ only in how the messaging loop
+ is done. <remark>[tbd: what is the messaging loop? Do we talk about it
+ anywyhere? is this too much info for a getting started guide?]</remark>
+
+zookeeperStarted.xml:217: The entry <emphasis
+ role="bold">syncLimit</emphasis> limits how far out of date a server can
+ be from a leader. [TBD: someone please verify that the previous is
+ true.]
+
+zookeeperStarted.xml:232: These are the "electionPort" numbers of the servers (as opposed to
+ clientPorts), that is ports for <remark>[tbd: feedback need: what are
+ these ports, exactly?]
+
+zookeeperStarted.xml:258: <remark>[tbd: what is the other config param?
+ (I believe two are mentioned above.)]</remark>
Propchange: hadoop/zookeeper/trunk/src/docs/src/documentation/TODO.txt
------------------------------------------------------------------------------
svn:eol-style = native
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/recipes.xml Wed Oct 1 15:28:35 2008
@@ -122,9 +122,6 @@
</listitem>
</orderedlist>
- <para><remark>[tbd: maybe an illustration would be nice for each of the
- recipes?]</remark></para>
-
<section id="sc_doubleBarriers">
<title>Double Barriers</title>
@@ -179,8 +176,7 @@
<listitem>
<para>if fewer children in L than<emphasis>
- x</emphasis>, wait for watch event <remark>[tbd: how do
- you wait?]</remark></para>
+ x</emphasis>, wait for watch event</para>
</listitem>
<listitem>
@@ -469,8 +465,7 @@
is deleted. In fact. that's valid behavior: as all those waiting
reader clients should be released since they have the lock. The herd
effect refers to releasing a "herd" when in fact only a single or a
- small number of machines can proceed. <remark>[tbd: myabe helpful to
- indicate which step this refers to?]</remark></para>
+ small number of machines can proceed.</para>
</note></para>
</section>
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml Wed Oct 1 15:28:35 2008
@@ -74,7 +74,7 @@
<section id="sc_systemReq">
<title>System Requirements</title>
- <para>Zookeeper runs in Java, release 1.6 or greater, as group of hosts
+ <para>Zookeeper runs in Java, release 1.5 or greater, as group of hosts
called a quorum. Three Zookeeper hosts per quorum is the minimum
recommended quorum size. At Yahoo!, Zookeeper is usually deployed on
dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB
@@ -87,7 +87,7 @@
<para>For reliable ZooKeeper service, you should deploy ZooKeeper in a
cluster known as a <firstterm>quorum</firstterm>. As long as a majority
of the quorum are up, the service will be available. Because Zookeeper
- requires a majority <remark>[tbd: why?]</remark>, it is best to use an
+ requires a majority, it is best to use an
odd number of machines. For example, with four machines ZooKeeper can
only handle the failure of a single machine; if two machines fail, the
remaining two machines do not constitute a majority. However, with five
@@ -101,7 +101,7 @@
<listitem>
<para>Install the Java JDK:</para>
- <screen>$yinst -i jdk-1.6.0.00_3 -br test <remark>[y! prop - replace with open equiv]</remark></screen>
+ <screen>$yinst -i jdk-1.6.0.00_3 -br test </screen>
</listitem>
<listitem>
@@ -109,15 +109,13 @@
swapping, which will seriously degrade Zookeeper performance. To
determine the correct value, load tests, make sure you are well
below the usage limit that would cause you to swap. Be conservative
- - use a maximum heap size of 3GB for a 4GB machine. <remark>[tbd:
- where would they do this? Environment variable,
- etc?]</remark></para>
+ - use a maximum heap size of 3GB for a 4GB machine.</para>
</listitem>
<listitem>
<para>Install the Zookeeper Server Package:</para>
- <screen>$ yinst install -nostart zookeeper_server <remark>[Y! prop - replace with open eq]</remark></screen>
+ <screen>$ yinst install -nostart zookeeper_server </screen>
</listitem>
<listitem>
@@ -168,8 +166,7 @@
<itemizedlist>
<listitem>
<para>In Java, you can run the following command to execute
- simple operations:<remark> [tbd: also, maybe give some of those
- simple operations?]</remark></para>
+ simple operations:</para>
<screen>$ java -cp zookeeper.jar:java/lib/log4j-1.2.15.jar:conf \
org.apache.zookeeper.ZooKeeperMain 127.0.0.1:2181</screen>
@@ -190,8 +187,7 @@
</itemizedlist>
<para>Running either program gives you a shell in which to execute
- simple file-system-like operations. <remark>[tbd: again, sample
- operations?]</remark> To connect to Zookeeper with the multithreaded
+ simple file-system-like operations. To connect to Zookeeper with the multithreaded
client, for example, you would run:</para>
<screen>$ cli_mt 127.0.0.1:2181</screen>
@@ -259,8 +255,7 @@
all the servers that make up a ZooKeeper server assuming the disk
layouts are the same. If servers use different configuration files,
care must be taken to ensure that the list of servers in all of the
- different configuration files match.<remark> [tbd: reformat in
- standard form, with legal values, etc]</remark></para>
+ different configuration files match.</para>
<section id="sc_minimumConfiguration">
<title>Minimum Configuration</title>
@@ -410,8 +405,7 @@
performance. (Note: The system property has no zookeeper
prefix, and the configuration variable name is different from
the system property. Yes - it's not consistent, and it's
- annoying.<remark> [tbd: is there any explanation for
- this?]</remark>)</para>
+ annoying.)</para>
</listitem>
</varlistentry>
@@ -450,9 +444,7 @@
election algorithm is not "0". When the election algorithm is
"0" a UDP port with the same port number as the port listed in
the <emphasis role="bold">server.num</emphasis> option will be
- used. <remark>[tbd: should that be <emphasis
- role="bold">server.id</emphasis>? Also, why isn't server.id
- documented anywhere?]</remark></para>
+ used.</para>
</listitem>
</varlistentry>
@@ -482,8 +474,7 @@
the leader can be configured to not accept clients and focus
on coordination. The default to this option is yes, which
means that a leader will accept client connections.
- <remark>[tbd: how do you specifiy which server is the
- leader?]</remark></para>
+ </para>
<note>
<para>Turning on leader selection is highly recommended when
@@ -501,9 +492,7 @@
<para>servers making up the Zookeeper quorum. When the server
starts up, it determines which server it is by looking for the
- file <filename>myid</filename> in the data directory.<remark>
- [tdb: should we mention somewhere about creating this file,
- myid, in the setup procedure?]</remark> That file contains the
+ file <filename>myid</filename> in the data directory. That file contains the
server number, in ASCII, and it should match <emphasis
role="bold">x</emphasis> in <emphasis
role="bold">server.x</emphasis> in the left hand side of this
@@ -516,9 +505,7 @@
<para>The port numbers <emphasis role="bold">nnnn</emphasis>
in this setting are the <emphasis>electionPort</emphasis>
numbers of the servers (as opposed to clientPorts).
- <remark>[tbd: is the next sentence explanation an of what the
- election port or is it a description of a special case?]
- </remark>If you want to test multiple servers on a single
+ If you want to test multiple servers on a single
machine, the individual choices of electionPort for each
server can be defined in each server's config files using the
line electionPort=xxxx to avoid clashes.</para>
@@ -534,9 +521,7 @@
<para>Amount of time, in ticks (see <ulink
url="#id_tickTime">tickTime</ulink>), to allow followers to
sync with ZooKeeper. If followers fall too far behind a
- leader, they will be dropped. <remark>[tbd: is this a correct
- rewording: if followers fall beyond this limit, they are
- dropped?]</remark></para>
+ leader, they will be dropped.</para>
</listitem>
</varlistentry>
</variablelist>
@@ -563,8 +548,7 @@
<para>Requires updates to be synced to media of the
transaction log before finishing processing the update. If
this option is set to no, ZooKeeper will not require updates
- to be synced to the media. <remark>[tbd: useful because...,
- dangerous because...]</remark></para>
+ to be synced to the media.</para>
</listitem>
</varlistentry>
@@ -593,7 +577,7 @@
<para>(Java system property: <emphasis
role="bold">zookeeper.skipACL</emphasis>)</para>
- <para>Skips ACL checks. <remark>[tbd: when? where?]</remark>
+ <para>Skips ACL checks.
This results in a boost in throughput, but opens up full
access to the data tree to everyone.</para>
</listitem>
@@ -662,12 +646,7 @@
<section id="sc_monitoring">
<title>Monitoring</title>
-
- <para><remark>[tbd: Patrick, Ben, et al: I believe the Message Broker
- team does perform routine monitoring of Zookeeper. But I might be
- wrong. To your knowledge, is there any monitoring of a Zookeeper
- deployment that will a Zookeeper sys admin will want to do, outside of
- Yahoo?]</remark></para>
+ <remark>[tbd]</remark>
</section>
<section id="sc_dataFileManagement">
@@ -773,8 +752,7 @@
but things will really act strange if clients have a list of
ZooKeeper servers that are in different ZooKeeper clusters. Also,
the server lists in each Zookeeper server configuration file
- should be consistent with one another. <remark>[tbd: I'm assuming
- this last part is true. Is it?]</remark></para>
+ should be consistent with one another.</para>
</listitem>
</varlistentry>
@@ -819,9 +797,7 @@
<title>Best Practices</title>
<para>For best results, take note of the following list of good
- Zookeeper practices. <remark>[tbd: I just threw this section in. Do we
- have list that is is different from the "things to avoid"? If not, I can
- easily remove this section.]</remark></para>
+ Zookeeper practices. <remark>[tbd...]</remark></para>
</section>
</chapter>
</book>
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperOver.xml Wed Oct 1 15:28:35 2008
@@ -291,11 +291,9 @@
<para><xref linkend="fg_zkComponents" /> shows the high-level components
of the ZooKeeper service. With the exception of the request processor,
- <remark>[tbd: where does the request processor live?]</remark> each of
+ each of
the servers that make up the ZooKeeper service replicates its own copy
- of each of components. <remark>[tbd: I changed the wording in this
- sentence from the white paper. Can someone please make sure it is still
- correct?]</remark></para>
+ of each of components.</para>
<para><figure id="fg_zkComponents">
<title>ZooKeeper Components</title>
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperProgrammers.xml Wed Oct 1 15:28:35 2008
@@ -139,8 +139,7 @@
</listitem>
<listitem>
- <para>The following characters are not allowed because <remark>[tbd:
- do we need reasons?]</remark> :\ud800 -uF8FFF, \uFFF0-uFFFF, \uXFFFE -
+ <para>The following characters are not allowed: \ud800 -uF8FFF, \uFFF0-uFFFF, \uXFFFE -
\uXFFFF (where X is an digit 1 - E), \uF0000 - \uFFFFF.</para>
</listitem>
@@ -170,8 +169,7 @@
it must supply the version of the data of the znode it is changing. If
the version it supplies doesn't match the actual version of the data,
the update will fail. (This behavior can be overridden. For more
- information see... <remark>[tbd... reference here to the section
- describing the special version number -1]</remark></para>
+ information see... )<remark>[tbd...]</remark></para>
<note>
<para>In distributed application engineering, the word
@@ -195,9 +193,7 @@
sends the client a notification. More information about watches can be
found in the section
<ulink url="#ch_zkWatches">Zookeeper Watches</ulink>.
- <remark>[tbd: fix this link] [tbd: Ben there is note from to emphasize
- that "it is queued". What is "it" and is what we have here
- sufficient?]</remark></para>
+ <remark>[tbd]</remark></para>
</section>
<section>
@@ -347,9 +343,7 @@
assigns to the client. If the client connects to a different ZooKeeper
server, it will send the session id as a part of the connection handshake.
As a security measure, the server creates a password for the session id
- that any ZooKeeper server can validate. <remark>[tbd: note from Ben:
- "perhaps capability is a better word." need clarification on that.]
- </remark>The password is sent to the client with the session id when the
+ that any ZooKeeper server can validate.The password is sent to the client with the session id when the
client establishes the session. The client sends this password with the
session id whenever it reestablishes the session with a new server.</para>
@@ -605,9 +599,9 @@
provide complete client synchronization. (For more information,
see the <ulink
url="recipes.html#sc_recipes_Locks">Locks</ulink>
- <remark>[tbd: fix final link target]</remark> in <ulink
+ <remark>[tbd:...]</remark> in <ulink
url="recipes.html">Zookeeper Recipes</ulink>.
- <remark>[tbd: fix final link target]</remark>).</para>
+ <remark>[tbd:..]</remark>).</para>
</listitem>
</varlistentry>
</variablelist>
@@ -778,8 +772,7 @@
<note>
<para>See INSTALL for general information about running
- <emphasis role="bold">configure</emphasis>. <remark>[tbd: what
- is INSTALL? a directory? a file?]</remark></para>
+ <emphasis role="bold">configure</emphasis>.</para>
</note>
</listitem>
@@ -819,8 +812,7 @@
<para>To verify that the node's been created:</para>
<para>You should see a list of node who are children of the root node
- "/". <remark>[tbd: document all the cli commands (I think this is
- Ben's tbd? It's from sourceforge)]</remark></para>
+ "/".</para>
<para>In order to be able to use the ZooKeeper API in your application
you have to remember to</para>
@@ -843,8 +835,7 @@
</orderedlist>
<para>Refer to <xref linkend="ch_programStructureWithExample"/>for examples of usage in Java and C.
- <remark>[tbd: some kind of short tutorial would be helpful, I guess
- (ben's tbd?) ][tbd: whatever the case, make sure that link points to something.]</remark></para>
+ <remark>[tbd]</remark></para>
</section>
</section>
</chapter>
@@ -852,9 +843,8 @@
<chapter id="ch_guideToZkOperations">
<title>Building Blocks: A Guide to ZooKeeper Operations</title>
- <para><remark>[Engineering input needed. This is a new section. The below
- is just placeholder, and was actually copied from the overview book. There
- should probably be a subsection on each of those operations, with a little
+ <para><remark>[tbd: This is a new section. The below
+ is just placeholder. Eventually, a subsection on each of those operations, with a little
bit of illustrative code for each op.] </remark></para>
<para>One of the design goals of ZooKeeper is provide a very simple
@@ -1054,8 +1044,7 @@
</varlistentry>
<varlistentry>
- <term><ulink url="recipes.html">Zookeeper Recipes [tbd: fix
- linkend for apache site]</ulink></term>
+ <term><ulink url="recipes.html">Zookeeper Recipes</ulink></term>
<listitem>
<para>Pseudo-level discussion of the implementation of various
@@ -1068,7 +1057,7 @@
<term><remark>[tbd]</remark></term>
<listitem>
- <para>Whatever good sources anyone can think of...</para>
+ <para>Any other good sources anyone can think of...</para>
</listitem>
</varlistentry>
</variablelist>
Modified: hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml
URL: http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml?rev=700955&r1=700954&r2=700955&view=diff
==============================================================================
--- hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml (original)
+++ hadoop/zookeeper/trunk/src/docs/src/documentation/content/xdocs/zookeeperStarted.xml Wed Oct 1 15:28:35 2008
@@ -70,20 +70,15 @@
<para>Zookeeper requires Java 1.5 or more recent.</para>
</note>
- <para>[tbd: should we start w/ a word here about were to get the source,
- exactly what to download, how to unpack it, and where to put it? Also,
- does the user need to be in sudo, or can they be under their regular
- login?]</para>
-
<para>Once you have downloaded the ZooKeeper source, cd to the root of
your ZooKeeper source, and run "ant jar". For example:<screen>$ cd ~/dev/zookeeper
$ ~/dev/zookeeper/: ant jar</screen></para>
<para>This should generate a JAR file called zookeeper.jar. To start
- Zookeeper, compile and run zookeeper.jar. <emphasis>[tbd, some more
+ Zookeeper, compile and run zookeeper.jar. <remark>[tbd, some more
instruction here. Perhaps a command line? Are these two steps or
- one?]</emphasis></para>
+ one?]</remark></para>
<para>To start ZooKeeper you need a configuration file. Here is a sample
file:</para>
@@ -136,7 +131,7 @@
<para>ZooKeeper logs messages using log4j -- more detail available in
the <ulink url="zookeeperProgrammers.html#Logging">Logging</ulink>
- section of the Programmer's Guide.<remark revision="include_tbd">[tbd:
+ section of the Programmer's Guide.<remark>[tbd:
real reference needed]</remark> You will see log messages coming to the
console and/or a log file depending on the log4j configuration.</para>
@@ -185,9 +180,7 @@
<para>ZooKeeper has a Java bindings and C bindings. They are
functionally equivalent. The C bindings exist in two variants: single
threaded and multi-threaded. These differ only in how the messaging loop
- is done. <remark>[tbd: what is the messaging loop? Do we talk about it
- anywyhere? is this too much info for a getting started guide?]</remark>
- For more information, see the <ulink
+ is done. For more information, see the <ulink
url="zookeeperProgrammers.html#ch_programStructureWithExample.html">Programming
Examples in the Zookeeper Programmer's Guide</ulink> for
sample code using of the different APIs.</para>
@@ -216,8 +209,7 @@
timeouts ZooKeeper uses to limit the length of time the Zookeeper
servers in quorum have to connect to a leader. The entry <emphasis
role="bold">syncLimit</emphasis> limits how far out of date a server can
- be from a leader. [TBD: someone please verify that the previous is
- true.]</para>
+ be from a leader.</para>
<para>With both of these timeouts, you specify the unit of time using
<emphasis role="bold">tickTime</emphasis>. In this example, the timeout
@@ -232,8 +224,7 @@
<para>Finally, note the "2888" port numbers after each server name.
These are the "electionPort" numbers of the servers (as opposed to
- clientPorts), that is ports for <remark>[tbd: feedback need: what are
- these ports, exactly?]</remark>.</para>
+ clientPorts), that is ports for <remark>[tbd]</remark>.</para>
<note>
<para>If you want to test multiple servers on a single machine, define
@@ -259,8 +250,7 @@
</listitem>
<listitem>
- <para><remark>[tbd: feedback need: what is the other config param?
- (I believe two are mentioned above.)]</remark></para>
+ <para><remark>[tbd: what is the other config param?]</remark></para>
</listitem>
</itemizedlist>
</section>