You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by bu...@apache.org on 2011/06/28 04:51:01 UTC
svn commit: r791701 [2/4] - in /websites/staging/zookeeper/trunk/content:
bookkeeper/ bookkeeper/docs/ bookkeeper/docs/trunk/ doc/trunk/
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigDesign.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigDesign.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigDesign.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,320 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - </title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Style</h1>
+
+<p>We have provided an Eclipse Formatter file <code>formatter.xml</code> with all the formatting conventions currently used in the project. Highlights include no tabs, 4-space indentation, and 120-char width. Please respect this so as to reduce the amount of formatting-related noise produced in commits.</p>
+
+<h1>Static Analysis</h1>
+
+<p>We would like to use static analysis tools <span class="caps">PMD </span>and FindBugs to maintain code quality. However, we have not yet arrived at a consensus on what rules to adhere to, and what to ignore.</p>
+
+<h1>Netty Notes</h1>
+
+<p>The asynchronous network IO infrastructure that Hedwig uses is <a href="http://www.jboss.org/netty">Netty</a>. Here are some notes on Netty's concurrency architecture and its filter pipeline design.</p>
+
+<h2>Concurrency Architecture</h2>
+
+<p>After calling <code>ServerBootstrap.bind()</code>, Netty starts a boss thread (<code>NioServerSocketPipelineSink.Boss</code>) that just accepts new connections and registers them with one of the workers from the <code>NioWorker</code> pool in round-robin fashion (pool size defaults to <span class="caps">CPU </span>count). Each worker runs its own select loop over just the set of keys that have been registered with it. Workers start lazily on demand and run only so long as there are interested fd's/keys. All selected events are handled in the same thread and sent up the pipeline attached to the channel (this association is established by the boss as soon as a new connection is accepted).</p>
+
+<p>All workers, and the boss, run via the executor thread pool; hence, the executor must support at least two simultaneous threads.</p>
+
+<h2>Handler Pipeline</h2>
+
+<p>A pipeline implements the intercepting filter pattern. A pipeline is a sequence of handlers. Whenever a packet is read from the wire, it travels up the stream, stopping at each handler that can handle upstream events. Vice-versa for writes. Between each filter, control flows back through the centralized pipeline, and a linked list of contexts keeps track of where we are in the pipeline (one context object per handler).</p>
+
+<h1>Distributed Performance Evaluation</h1>
+
+<p>We've included some scripts to repeatedly run varying configurations of Hedwig on a distributed testbed and collect the resulting data. The experiments use the <code>org.apache.hedwig.client.App</code> client application and are driven by <code>scripts/hw.bash</code> (via the <code>app</code> command).</p>
+
+<p>Currently, we have two types of experiments: subscription benchmarks and publishing benchmarks.</p>
+
+<h2>Subscription Benchmarks</h2>
+
+<p>The subscription benchmark compares synchronous and asynchronous subscriptions. Because the synchronicity of subscriptions is a server configuration parameter, the servers must be restarted to change this. The benchmarks varies the maximum number of outstanding subscription requests.</p>
+
+<p>To run the subscription benchmark with wilbur6 as the subscriber and wilbur1 as its default hub:</p>
+
+<p>hosts=wilbur6 scripts/hw.bash sub-exp wilbur1</p>
+
+<p>This produces log files into the <code>sub</code> directory, which may then be analyzed using the analysis scripts.</p>
+
+<h2>Publishing Benchmarks</h2>
+
+<p>The publishing benchmark measures the throughput and latency of publishing messages within a <span class="caps">LAN </span>and across a <span class="caps">WAN.</span> It varies the following parameters:</p>
+
+<ul>
+<li>maximum number of outstanding publish requests</li>
+<li>number of publishers</li>
+<li>number of (local) receivers</li>
+</ul>
+
+<p>We vary each dimension separately (and have default settings) to avoid a combinatorial explosion in the number of configurations to explore.</p>
+
+<p>First, start a (standalone) instance:</p>
+
+<p>scripts/hw.bash start-region '' $hwhost $zkhost $bk1host $bk2host $bk3host</p>
+
+<p>To run this over <code>$host1</code> through <code>$host3</code>, with the number of publishers/subscribers varying linearly over this set:</p>
+
+<p>npars="20 40 60 80 100" scripts/hw.bash pub-exps "$host1 $host2 $host3" $hwhost $zkhost</p>
+
+<p>This will vary the number of outstanding publish requests as specified in <code>npars</code>.</p>
+
+<p>You may also optionally run this experiment with a second subscribing region:</p>
+
+<p>scripts/hw.bash start-zk-bks $zkhost $bk1host $bk2host <span class="math">bk3host npars="..." scripts/hw.bash pub-exps "</math>host1 $host2 $host3" $hwhost $zkhost $rrecv $rhwhost $rzkhost</p>
+
+<p>where the final three extra arguments specify the client receiver, Hedwig, and Zookeeper hosts, in that order.</p>
+
+<p>This command will produce files into <code>./pub/</code>, which can then be process using <code>analyze.py</code>.</p>
+
+<h1>Analysis and Visualization</h1>
+
+<p><code>scripts/analyze.py</code> produces plots from the collected experimental data. It has just a few immediate dependencies. In the following, the indentation signifies nested dependencies, like an upside-down tree:</p>
+
+<pre><code> component AAA that component AA requires
+ component AAB that component AA requires
+component AA that component A requires
+ component ABA that component AB requires
+ component ABB that component AB requires
+component AB that component A requires</code></pre>
+
+
+<p>component A that analysis tools depend on component <span class="caps">BAA </span>that component BA requires component <span class="caps">BAB </span>that component BA requires component BA that component B requires component <span class="caps">BBA </span>that component BB requires component <span class="caps">BBB </span>that component BB requires component BB that component B requires component B that analysis tools depend on</p>
+
+<p>The reason the tree is upside-down is so that you can treat this whole thing as a chunk of bash script.</p>
+
+<p><a href="http://toastball.net/toast/">toast</a> is a utility that makes it a breeze to install all this software, but you do need to make sure your environment is set up correctly (e.g. <code>PKG_CONFIG_PATH</code> must point to <code>~/.toast/armed/lib/pkgconfig/</code>).</p>
+
+<p>Setup:</p>
+
+<p>wget <del>O</del> http://toastball.net/toast/toast|perl -x - arm toast</p>
+
+<p>toast arm "http://www.python.org/ftp/python/2.6.2/Python-2.6.2.tar.bz2"</p>
+
+<p>toast arm numpy</p>
+
+
+
+<pre>
+ toast arm libpng
+
+ toast arm pixman
+
+ toast arm freetype
+
+ toast arm 'ftp://xmlsoft.org/libxml2/libxml2-2.7.3.tar.gz'
+
+ toast arm fontconfig
+
+ toast arm cairo
+
+toast arm pycairo
+</pre>
+
+
+
+<p>hg clone https://yang@bitbucket.org/yang/pycha/ pycha/setup.bash -d -p $path_to_install_to</p>
+
+<p>svn co https://assorted.svn.sourceforge.net/svnroot/assorted/python-commons/trunk/ python-commons/ python-commons/setup.bash -d -p $path_to_install_to</p>
+
+<p>To analyze the publishing experiments, change to the <code>pub</code> data directory and run:</p>
+
+<p>scripts/analyze.py pub</p>
+
+<p>To analyze the subscription experiments, change to the <code>sub</code> data directory and run:</p>
+
+<p>scripts/analyze.py sub</p>
+
+<h1>Debugging</h1>
+
+<p>You can attach an Eclipse debugger (or any debugger) to a Java process running on a remote host, as long as it has been started with the appropriate <span class="caps">JVM </span>flags. (See the Building Hedwig document to set up your Eclipse environment.) To launch something using <code>hw.bash</code> with debugger attachment enabled, prefix the command with <code>attach=true</code>, e.g.:</p>
+
+<p>attach=true scripts/hw.bash start-regions myregions.cfg</p>
+
+<h1>Profiling</h1>
+
+<p>The scripts we have provided include ways for you to launch with YourKit profiling enabled.</p>
+
+<p>To deploy YourKit onto a number of machines:</p>
+
+<p>hosts="..." scripts/hw.bash setup-yjp $path_to_yjp</p>
+
+<p>where the path points to the <a href="http://www.yourkit.com/download/yjp-8.0.15.zip">YourKit Linux zip archive</a> (which is freely available and doesn't require any license to use).</p>
+
+<p>Now when using the scripts to run distributed experiments, to profile anything with YourKit, prefix the command with <code>use_yjp=true</code>. E.g.:</p>
+
+<p>use_yjp=true scripts/hw.bash start-regions regions.cfg</p>
+
+<p>Now you may start on your local machine the YourKit <span class="caps">GUI </span>and connect to the hosts that you're interested in.</p>
+
+<p>Note that you may want to disable the default set of filters in YourKit.</p>
+
+<h1>Pseudocode</h1>
+
+<p>This summarizes the control flow through the system.</p>
+
+<p>publishhandler topicmgr.getowner (maybe) claim the topic, calling back into persmgr.acquiredtopic read /hedwig/standalone/topics/TOPIC (which should initially be empty) for each line, parse as "STARTSEQ" # <span class="caps">TODO </span>how is this written? ledger = bk.openledger(ledgerid) lastid = ledger.getlast if lastid > 0, lrs[startseq] = persmgr.ledger2lr[ledgerid] = new LedgerRange(ledger, ledgerid, startseq, startseq + lastid # <span class="caps">TODO </span>what are ledger ranges? create new ledger for topic # <span class="caps">TODO </span>read lr = new LedgerRange(ledger, ledgerid, lastid, -1) lrs[lastid] = lr persmgr.topic2ranges[topic] = lrs add region info to pub req and send that to persmgr.persistmessage entryid = persmgr.topic2ranges[topic].last.ledger.addentry(the pub'd data) update persmgr.topic2lastseq[topic]: .local = persmgr.ledger2lr[ledger id].startseq + entryid .regions = maxes of orig seq and incoming pub seq</p>
+
+<p>subscribehandler topicmgr.getowner... delivmgr.startservingsubscription(topic, endpoint, ishubsubscriber) delivmgr.endpoint2sub[endpoint] = new subscriber(lastseq = persmgr.getcurrentseqidfortopic(topic).local) delivmgr.topic2ptr2subs[topic][ptr].add(sub) sub.delivernextmessage sub.curseq = persmgr.getseqidafterskipping(topic, sub.lastseq, skip = 1) msg = persmgr.scansinglemessage(topic, seq = sub.curseq) if persmgr.topic2lastseq[topic].local >= seq lr = persmgr.topic2ranges[topic].floor(seq) return lr.ledger.read(first = last = seq - lr.startseq) if failed, then retry in 1 s endpoint.send(msg) movedeliveryptr delivmgr.topic2ptr2subs[topic][sub.lastseq].remove(sub) delivmgr.topic2ptr2subs[topic][sub.curseq].add(sub) previd = sub.lastseq, sub.lastseq = sub.curseq sub.delivernextmessage...</p>
+
+<h1>ReadAhead Cache</h1>
+
+<p>The delivery manager class is responsible for pushing published messages from the hubs to the subscribers. The most common case is that all subscribers are connected and either caught up, or close to the tail end of the topic. In this case, we don't want the delivery manager to be polling bookkeeper for any newly arrived messages on the topic; new messages should just be pushed to the delivery manager. However, there is also the uncommon case when a subscriber is behind, and messages must be pulled from Bookkeeper.</p>
+
+<p>Since all publishes go through the hub, it is possible to cache the recently published messages in the hub, and then the delivery manager won't have to make the trip to bookkeeper to get the messages but instead get them from local process memory.</p>
+
+<p>These ideas of push, pull, and caching are unified in the following way: - A hub has a cache of messages</p>
+
+<ul>
+<li>When the delivery manager wants to deliver a message, it asks the cache for it. There are 3 cases:</li>
+<li>The message is available in the cache, in which case it is given to the delivery manager</li>
+<li>The message is not present in the cache and the seq-id of the message is beyond the last message published on that topic (this happens if the subscriber is totally caught up for that topic). In this case, a stub is put in the cache in order to notify the delivery manager when that message does happen to be published.</li>
+<li>The message is not in the cache but has been published to the topic. In this case, a stub is put in the cache, and a read is issued to bookkeeper.</li>
+<li>Whenever a message is published, it is cached. If there is a stub already in the cache for that message, the delivery manager is notified.</li>
+<li>Whenever a message is read from bookkeeper, it is cached. There must be a stub for that message (since reads to bookkeeper are issued only after putting a stub), so the delivery manager is notified.</li>
+<li>The cache does readahead, i.e., if a message requested by the delivery manager is not in the cache, a stub is established not only for that message, but also for the next n messages where n is configurable (default 10). On a cache hit, we look ahead n/2 messages, and if that message is not present, we establish another n/2 stubs. In short, we always ensure that the next n stubs are always established.</li>
+<li>Over time, the cache will grow in size. There are 2 pruning mechanisms:</li>
+<li>Once all subscribers have consumed up to a particular seq-id, they notify the cache, and all messages up to that seq-id are pruned from the cache.</li>
+<li>If the above pruning is not working (e.g., because some subscribers are down), the cache will eventually hit its size limit which is configurable<br />
+ (default, half of maximum jvm heap size). At this point, messages are just pruned in <span class="caps">FIFO </span>order. We use the size of the blobs in the message for estimating the cache size. The assumption is that that size will dominate over fixed, object-level size overheads.</li>
+<li>Stubs are not purged because according to the above simplification, they are of 0 size.</li>
+</ul>
+
+<h1>Scalability Bottlenecks Down the Road</h1>
+
+<ul>
+<li>Currently each topic subscription is served on a different channel. The number of channels will become a bottleneck at higher channels. We should switch to an architecture, where multiple topic subscriptions between the same client, hub pair should be served on the same channel. We can have commands to start, stop subscriptions sent all the way to the server (right now these are local).</li>
+<li>Publishes for a topic are serialized through a hub, to get ordering guarantees. Currently, all subscriptions to that topic are served from the same hub. If we start having large number of subscribers to heavy-volume topics, the outbound bandwidth at the hub, or the <span class="caps">CPU </span>at that hub might become the bottleneck. In that case, we can setup other regions through which the messages are routed (this hierarchical scheme) reduces bandwidth requirements at any single node. It should be possible to do this entirely through configuration.</li>
+</ul>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,289 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - </title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Design</h1>
+
+<p>In Hedwig, clients publish messages associated with a topic, and they subscribe to a topic to receive all messages published with that topic. Clients are associated with (publish to and subscribe from) a Hedwig <em>instance</em> (also referred to as a <em>region</em>), which consists of a number of servers called <em>hubs</em>. The hubs partition up topic ownership among themselves, and all publishes and subscribes to a topic must be done to its owning hub. When a client doesn't know the owning hub, it tries a default hub, which may redirect the client.</p>
+
+<p>Running a Hedwig instance requires a Zookeeper server and at least three Bookkeeper servers.</p>
+
+<p>An instance is designed to run within a datacenter. For wide-area messaging across datacenters, specify in the server configuration the set of default servers for each of the other instances. Dissemination among instances currently takes place over an all-to-all topology. Local subscriptions cause the hub to subscribe to all other regions on this topic, so that the local region receives all updates to it. Future work includes allowing the user to overlay alternative topologies.</p>
+
+<p>Because all messages on a topic go through a single hub per region, all messages within a region are ordered. This means that, for a given topic, messages are delivered in the same order to all subscribers within a region, and messages from any particular region are delivered in the same order to all subscribers globally, but messages from different regions may be delivered in different orders to different regions. Providing global ordering is prohibitively expensive in the wide area. However, in Hedwig clients such as <span class="caps">PNUTS, </span>the lack of global ordering is not a problem, as <span class="caps">PNUTS </span>serializes all updates to a table row at a single designated master for that row.</p>
+
+<p>Topics are independent; Hedwig provides no ordering across different topics.</p>
+
+<p>Version vectors are associated with each topic and serve as the identifiers for each message. Vectors consist of one component per region. A component value is the region's local sequence number on the topic, and is incremented each time a hub persists a message (published either locally or remotely) to <span class="caps">BK.</span></p>
+
+<p><span class="caps">TODO</span>: More on how version vectors are to be used, and on maintaining vector-maxes.</p>
+
+<h1>Entry Points</h1>
+
+<p>The main class for running the server is <code>org.apache.hedwig.server.netty.PubSubServer</code>. It takes a single argument, which is a <a href="http://commons.apache.org/configuration/">Commons Configuration</a> file. Currently, for configuration, the source is the documentation. See <code>org.apache.hedwig.server.conf.ServerConfiguration</code> for server configuration parameters.</p>
+
+<p>The client is a library intended to be consumed by user applications. It takes a Commons Configuration object, for which the source/documentation is in <code>org.apache.hedwig.client.conf.ClientConfiguration</code>.</p>
+
+<p>We have provided a simple client application, <code>org.apache.hedwig.client.App</code>, that can drive a number of benchmarks. This also takes a single configuration file argument, which is fed to the client library.</p>
+
+<p>We've provided a number of scripts to faciliate running servers and clients in a variety of configurations, including over distributed hosts. These are all consolidated in <code>scripts/hw.bash</code>. Although the snippets in this documentation run the script from the hedwig main directory, you can run it from any location. Apologies in advance for these being bash scripts; time permitting, a more robust and maintainable support/tooling infrastructure would be ideal.</p>
+
+<h1>Deployment</h1>
+
+<p>When ssh-ing into a new host, you are requested to verify and accept the host key. In order to automatically accept the host keys for many new hosts (dangerous), use:</p>
+
+<p>hosts="$host1 $host2 ..." scripts/hw.bash warmup</p>
+
+<p>The <code>hosts</code> variable is set here to the list of hosts that you would like to warm up.</p>
+
+<p>To set up <span class="caps">JDK6 </span>on some hosts, use:</p>
+
+<p>hosts="..." scripts/hw.bash setup-java $path_to_modified_jdk6</p>
+
+<p>The argument must point to a <span class="caps">JDK6 </span>binary self-extracting executable, but with the <code>more</code> command that displays the License agreement replaced with <code>cat</code>. Unfortunately, this step must be performed manually. This script will extract the <span class="caps">JDK </span>directly into the home directory and update <code>$PATH</code> in <code>~/.bashrc</code> (in an idempotent fashion).</p>
+
+<p>Because the current implementation uses a single socket per subscription, the Hedwig launching scripts all require a high <code>ulimit</code> on the number of open file descriptors. Non-root users can only use up to the limit specified in <code>/etc/security/limits.conf</code>; to raise this to 1024^2, run:</p>
+
+<p>hosts="..." scripts/hw.bash setup-limits</p>
+
+<p>This uses <code>ssh</code> so that you need to enter your password for <code>sudo</code> just once.</p>
+
+<p>For most of the commands presented in the next section, you may prefix the command with:</p>
+
+<p>push_jar=true ...</p>
+
+<p>to first push the assembly jar (assumed to be available in <code>server/target/</code>) to all hosts.</p>
+
+<h1>Running Servers</h1>
+
+<p>To start three BK bookies on ports 3181-3183 on localhost (directories must all exist):</p>
+
+<p>scripts/hw.bash bk 3181 $bk1_journal_dir $bk1_ledger_dir & scripts/hw.bash bk 3182 $bk2_journal_dir $bk2_ledger_dir & scripts/hw.bash bk 3183 $bk3_journal_dir $bk3_ledger_dir &</p>
+
+<p>To start a ZK on port 2181 (directory must exist):</p>
+
+<p>scripts/hw.bash zk 2181 /path/for/zk/ &</p>
+
+<p>To register the BKs with the ZK (so that Hedwig knows where to find the bookies):</p>
+
+<p>scripts/hw.bash setup-bk localhost:2181 <code>hostname</code>:3181 <code>hostname</code>:3182 <code>hostname</code>:3183</p>
+
+<p>Everything up to this point may be done using a single command over a set of hosts, with ZK on port 9877 and BK on port 9878. The following function takes 2 arguments. The first is the ZK host. The second is a string list of BK hosts:</p>
+
+<p>scripts/hw.bash start-zk-bks <span class="math">zkhost "</math>bk1host $bk2host $bk3host ..."</p>
+
+<p>Note that the hosts may be <span class="caps">SSH </span>profile aliases in your <code>~/.ssh/config</code>; the script will parse this file and look up their hostnames where necessary. This applies for the hosts specified in the other commands.</p>
+
+<p>Also, the scripts use the <code>bk-journal</code> and <code>bk-ledger</code> functions in <code>hw.bash</code> to determine where to place the BK journal and ledger, given a hostname.</p>
+
+<p>To start a Hedwig server locally:</p>
+
+<p>scripts/hw.bash hw server.conf &</p>
+
+<p>To start Hedwig servers on some hosts "$hw1host $hw2host $hw3host ..." on port 9876, using $zkhost as the ZK server:</p>
+
+<p>scripts/hw.bash start-hw '' "$hw1host $hw2host $hw3host ..." $zkhost</p>
+
+<p>Above, the first empty string argument is the list of default servers to each of the other regions. You may run multiple connected instances of Hedwig this way.<br />
+E.g., to start three regions each with a single Hedwig hub that talk to each other, and using the hw.bash default server ports of 9875 (non-SSL) and 9876 (SSL):</p>
+
+<p>scripts/hw.bash start-hw "$hw2host:9875:9876 <span class="math">hw3host:9875:9876" "</math>hw1host" <span class="math">zk1host scripts/hw.bash start-hw "</math>hw1host:9875:9876 <span class="math">hw3host:9875:9876" "</math>hw2host" <span class="math">zk2host scripts/hw.bash start-hw "</math>hw1host:9875:9876 <span class="math">hw2host:9875:9876" "</math>hw3host" $zk3host</p>
+
+<p>Everything up to this point may be done using a single command over a set of hosts:</p>
+
+<p>scripts/hw.bash start-region '' "$hw1host $hw2host $hw3host ..." <span class="math">zkhost "</math>bk1host $bk2host $bk3host ..."</p>
+
+<p>The first three arguments are the same as for <code>start-hw</code>.</p>
+
+<p>You may start multiple regions as well:</p>
+
+<p>scripts/hw.bash start-regions regions.cfg</p>
+
+<p>"regions.cfg" is a list of all regions, one per line, with each region having the following format:</p>
+
+<p>region=<Region name>, hub=<list of hub servers>, default=<single hub server>, zk=<single ZK server>, bk=<list of BK servers></p>
+
+<p>This will create all of the regions with an all-to-all topology. Each region is connected to the default hub server of every other region. The ", " delimiter is used to separate out the different parts of a region along with the hard-coded parameter names. There also needs to be a newline after the last region line. Here is an example file specifying three regions:</p>
+
+<p>region=wilbur, hub=wilbur90 wilbur91, default=wilbur90, zk=wilbur93, bk=wilbur93 wilbur94 wilbur95 region=re1, hub=sherpa7 sherpa8, default=sherpa7, zk=sherpa9, bk=sherpa9 sherpa10 sherpa11 region=peanuts, hub=peanuts1 peanuts2, default=peanuts2, zk=peanuts3, bk=peanuts3 peanuts4 peanuts5</p>
+
+<h1>Running the Client</h1>
+
+<p>To run the test client:</p>
+
+<p><span class="caps">JAVAFLAGS</span>="..." scripts/hw.bash hwc $conf_path</p>
+
+<p>where <code>$conf_path</code> is a client configuration file.</p>
+
+<p>To run the test client on some other hosts:</p>
+
+<p>hosts="..." <span class="caps">JAVAFLAGS</span>="..." scripts/hw.bash app $hwhost</p>
+
+<p>This will generate a simple configuration file assuming $hwhost is listening on the default <span class="caps">SSL </span>and non-SSL ports which are specified as global variables in hw.bash. Currently these are 9875 for non-SSL and 9876 for <span class="caps">SSL.</span></p>
+
+<p>Client usage is currently documented in the source. To run a subscription benchmark, set <code>JAVAFLAGS</code> to:</p>
+
+<p>-Dmode=sub -Dcount=10000 -Dnpar=100 -Dstart=5 -Dnwarmups=30</p>
+
+<p>This will first create 30 warm-up subscriptions to topics "warmup-5" through "warmup-34", then 10,000 benchmarked subscriptions to topics "topic-5" through "topic-10,004". It will have a pipeline depth of 100 requests, meaning that there will be at most 100 outstanding (unresponded) messages in flight at any moment.</p>
+
+<p>To run a publishing benchmark, set <code>JAVAFLAGS</code> to:</p>
+
+<p>-Dmode=pub -Dcount=10000 -Dnpar=100 -Dstart=5</p>
+
+<p>This will publish 10,000 messages to topic "topic-5", with a pipeline depth of 100 requests.</p>
+
+<p>At the end, the programs will print throughput and latency information.</p>
+
+<h1>Utilities</h1>
+
+<p>To kill all the user's Java processes running on some machines, use:</p>
+
+<p>hosts="..." scripts/hw.bash dkill</p>
+
+<p>To check if any processes are running and are using ports of interest (esp. 9876-9878):</p>
+
+<p>hosts="..." scripts/hw.bash dstatus</p>
+
+<p>Add an argument to <code>dstatus</code> (may be anything) to get a more detailed listing.</p>
+
+<p>To check if there's anything consuming the <span class="caps">CPU </span>on some machines:</p>
+
+<p>hosts="..." scripts/hw.bash tops</p>
+
+<p>To run an arbitrary command on multiple hosts in parallel:</p>
+
+<p>hosts="..." scripts/hw.bash parssh $command</p>
+
+<p>To do this in sequence:</p>
+
+<p>hosts="..." xargs= scripts/hw.bash parssh $command</p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,150 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - BookKeeper Documentation</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Apache BookKeeper documentation</h1>
+
+<ul>
+<li><a href="./bookkeeperOverview.html">Overview</a></li>
+<li><a href="./bookkeeperStarted.html">Getting started</a></li>
+<li><a href="./bookkeeperProgrammer.html">Programmer's Guide</a></li>
+<li><a href="./bookkeeperConfig.html">Admin Guide</a></li>
+<li><a href="./bookkeeperStream.html">Using BookKeeper stream library</a></li>
+</ul>
+
+<h1>Apache Hedwig documentation</h1>
+
+<ul>
+<li><a href="./hedwigBuild.html">Building Hedwig, or how to set up Hedwig</a></li>
+<li><a href="./hedwigUser.html">User's Guide, or how to program against the Hedwig <span class="caps">API </span>and how to run it</a></li>
+<li><a href="./hedwigDesign.html">Developer's Guide, or Hedwig internals and hacking details</a></li>
+</ul>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/index.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/index.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/index.html Tue Jun 28 02:51:00 2011
@@ -91,12 +91,12 @@ Learn more about Hedwig on the <a href="
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -105,9 +105,9 @@ Learn more about Hedwig on the <a href="
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/irc.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/irc.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/irc.html Tue Jun 28 02:51:00 2011
@@ -75,12 +75,12 @@
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -89,9 +89,9 @@
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/lists.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/lists.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/lists.html Tue Jun 28 02:51:00 2011
@@ -112,12 +112,12 @@ In order to post to the list, it is nece
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -126,9 +126,9 @@ In order to post to the list, it is nece
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html Tue Jun 28 02:51:00 2011
@@ -87,12 +87,12 @@
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -101,9 +101,9 @@
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/releases.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/releases.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/releases.html Tue Jun 28 02:51:00 2011
@@ -73,12 +73,12 @@
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -87,9 +87,9 @@
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/svn.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/svn.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/svn.html Tue Jun 28 02:51:00 2011
@@ -89,12 +89,12 @@
<h3>Project</h3>
<ul>
-<li><a href="./releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
-<li><a href="./credits.html">Credits</a></li>
-<li><a href="./bylaws.html">Bylaws</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
-<li><a href="./privacy.html">Privacy Policy</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
@@ -103,9 +103,9 @@
<h3>Developers</h3>
<ul>
-<li><a href="./lists.html">Mailing Lists</a></li>
-<li><a href="./irc.html"><span class="caps">IRC</span> Channel</a></li>
-<li><a href="./svn.html">Version Control</a></li>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
</ul>
<!-- sidenav -->
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.html Tue Jun 28 02:51:00 2011
@@ -273,7 +273,7 @@ document.write("Last Published: " + docu
<p>
<span class="codefrag computeroutput">
java -cp .:./zookeeper-<version>-bookkeeper.jar:./zookeeper-<version>.jar\
- :../log4j/apache-log4j-1.2.15/log4j-1.2.15.jar -Dlog4j.configuration=log4j.properties\
+ :lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar -Dlog4j.configuration=log4j.properties\
org.apache.bookkeeper.proto.BookieServer 3181 127.0.0.1:2181 /path_to_log_device/\
/path_to_ledger_device/
</span>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.pdf
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.pdf (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.pdf Tue Jun 28 02:51:00 2011
@@ -102,10 +102,10 @@ endobj
>>
endobj
22 0 obj
-<< /Length 2260 /Filter [ /ASCII85Decode /FlateDecode ]
+<< /Length 2282 /Filter [ /ASCII85Decode /FlateDecode ]
>>
stream
-Gasar>Ar7S'Roe[&GjfbL`)>d3nOWnlPaj]TPM2qD&isPU2;3>kYb=6m+:]&2]g[P@cYM(aE.=*NO`@K0&K//oni"92[e6j/u[gI+*CU:m"N/DW#%M\deW&8-/U^U(9%0j2nWkqC9':2Rm].s]0\U"8(tZUI4`CTYFP'le9d2%^CN)<\6]q[>6FVLSb4XoT@]")+Jk7nB$j99q'6G;a6sKk".'Xe1.QBJDXkd,>3Qk`Pk@)+qCtV?W)moNbA5?W'5:\AWmNo-W1#!8d$_Q##[&KK/h4IK>E8,9M,l:phHYBeCR_44`=oOBhX#(P6;I<+W+4ME]Mt[gCROJ!0k#^GH^U4WbV@9DdED;00*LdFYt.;lGSKJt844msPXO82+<FPVS4W%Zg_Us"(bAZ$7lFD[Z;&jr\dQeKHu/WaX+7^4rl:OFLpV4^<(Q._hb15>8QcrYj51Jl$@2/)(44NQAuA&$#TltX:X6;.c8ZZI5dgk\8W<k9Hug9G3hgpqcDMUo_*Uap+Y\"0(ceGQ0h).(j$r";,Ru$]i8"%b2-cDNB9IK`Tk24J[^&&*U\G&>ni\91*"c:-i\Q(M^aQ>!W5\]pR$c(KopifEH,VF]QJX-B^a&mf&d@W5JZ"E0i0uXNp`M"\jI2r046FsDq0SFG(^/d3-ls$d:Al7\?V1h(WeM3ncR.bhLS@,K0e#Ja4Y%`^*b\:R0lK(E0g2?cnJHM0Xb]c<l)5+V%/%n+.%R:R9V:U)QZIP8\1#)]o6a>*`()'i%1Hf[h*,,`npXKNOB`O$g>[QNj&6t>>d/B"7D-l;r,&JgoeKWSQUEAA_86achMFU"La8IDM%;<T9r/d9\UP)]r+JO3cp(5U[W7bs:bD7Meu6&L+`Qj+X6ne2S.s`N>[:,VpFOO^KO!0X\fZZ[NiHWR\Lc5V:V2A"%SXq\X,$d:*tkc\:^+ge"^i3lc3fAa$5*\\i*Bd7/<&#."$/%0;:X>j;5.s$6D)2
.%Yjj+`Tq^qG54:f;TS]'D#.HD/Irf*kol,e-IS]o=!\NS$mGu2L&n&u7)H8a$=3=^d[p'&<JF?ojH_'^NoUZQGj)sJ+Mm.Y)TLAtJIIsm1`lD5,?bPi!U?t`Ku),#d[Tm\/]`T/3Z8`HG.9W8GuGZ6g:Maf+3GA\'jJ`t/JdcSW\*Y=9!DK*i<IG@h!si-,3NnNGqCK-5$H4i`A%ZgF+E=WmpDp$iIc#nc_2q=jN*;lq_KWbnJ%;&Xnej5'&kh>:>g3tAJjSTN"V_iC"L(.AbW=^MjhC#4C/$\L9APSnWD4kSaI2ho4N@O%uu?pYBtn``92f<k=(%-6W`he@j(9KjB-pVS$@bMejG$:]M`,UJ9Zagq\Ip'^g*:k&7BO:]N;$K/"S@%cW@!Z34:>t,P37!/80^>c5fV6fint>\,pN?m5IMfR`e4X--pSoY@lt5Ts(c6!GKTc`Ubuf\tR`W\\tM;bFh`B\f'N?LYtC/L2;7qk?*suj(XEY(JqP379LOJ.Im]G'=b9'U(@S!qpcBN.m%Vd[,0c?d#b@s[j/1PpGN\h0IHdDK3XP^s$50g@4o;rY(MQ0EJ[<.I3S'6e<eLRXXo&h#LaD2Q\F)gpQp;2>'>h#C?B2KM&2&BQ(6/W+IDQ..g-(S9tkIUQ\'\MpE8dQKFOSb7P6*uPP=Hu&kFj2>^,VNmBEZ$lHu!a'kIu.+,!t*DVr;@<182U\PJ[MQUZ?';86Z>0i.VAP&_hG^[l9bj`k>,s%M:D.pmYpQ&WndR;DP2L*JmCPEmNI'`f<>Di7fW7fn6OEj-Ub8#I,T,8rOR5r32B/OOOu[lDB8ikd?/\.p^_*oO$WWVIO/Al0t,\Ek0R4=tS2)BfJ.K>LZ<42!2V6X3A7hheqDP*01?H5dJ9p9DgHI4E6tHH'WQ72hbX/nTo,G-*MWessYI73;cK%`4EE2cM`J_b=Pg$Y/R#dDjjcLO8p(/XkI>8t4A+^6l7X5(V:B2u(
N][(%]tZ[:=p\b`fk@.oJ/0iBJ<,Ej6E'3n]MVp.Kj0e$:278k?UYP4d6k20Ni][RB2F.e!?4QKG-0Gn7CB_3]Wfd>l1K%Vef`oI'*GH2geof;[(1sD`L^;6R^7&")QmsoDN]"?5>h9f`k!YEM44U9DG*"B3%9X`a7`fP5jmB\?;Zn1bM1m:h9C9'%-@NX)>Re<;kr^l8,?"`(D<n@,k!if>GF^r*gpC?I)H4oVs[pmf+L6UH]]GE0<S]XM,?0aZ1007rI>4W6:)>]E6s#kZO$N~>
+Gasar>Ar7S'Roe[&GjfbL`)>d3nOWnlPaj]TPM2qD&isPU2;3>kYb=6m+:]&2]g[P@cYM&aE.=*3?OOPp#>OjjA;JYc?cg9SaD"i0)SRZjhb3bH\?^=ZS%euhGEd6ZXeVjIriMs-Ejk(iO2<g_...@Zl>%Jf/f7fK'D8FO5P]97A^?2H#cq%92P
VfZ_0NT;;8Xh+:k#c_V>]IM&'`(BW],,;q]LdGI?\'_\M:6Bjb>!GrZ-l:3*^$_tM;=-WEIs/[u7$6cl(l$Ob#@),,?2?4%)h`r#c(Bk.V5!qI#P=P7%E!+%aJAP^X+8JY-itc/,O,c[7RN6!q2/d6I:clk@r(si#YLlIuTNIR]<Em\I8r-YW@01!QGPlfOa>k5,?V?9eQe$,#ED1s=*+4'4rMpVG5%)Q\%^bkjUGP=)c_11];qUfPk*m,jLhYk>_!MKK0^_FJ,s5tlNkSt8pT<5/R_N)i?3'SOY/]Jk;:(rKh'l%\cYrss[Y9:9\NrdGF%XUH,/Sq_Q$h4$#^e3a2PA-',%?)qOPh'a.Bj4G4@8+WMdm0'Y0<@H?uh12=/[#fSh>uG=/NgD[O[@goV?@UBEVB!NhMj"XY2;O!c<J])ct^]b=CDHcZ/'MBC3O3YI_=HBZ[e2o/9[]9<U?sM:FPO_5%us1Tj4fMZ6crMXc7i'2eagV_.nZJ@\H)b0kf5OZs\M^k5>qKfD?SYGO?afl>2cHAA7_6amQ]:<53HgHr]mR/#\JE^_V\"(/Zp\bod1%P%=q,oRfuM6q2!Da&ZZWX&Z39NOZ3G?G_$s*gC&o-Kb3S6lTQl\4!fpQQ]3lW0@0oDsf^:TQKVjti"Ijh'pdJ\1q/hmTiV:-\Adck+%g"opJi1/-[6)"ngS+rWbM4(R,i0\<T'BMA(4AJ'J8KO1p$h4bBaRaQAi:R^++7XP58^L[6s:\gimEYPG/o5)V3\7L939M;-2?RP]8:!'H(7eURd,MnLYRACEB!OrNM/S]jO*sc,S6hJa(3fOtmd?l\%SKZ0e'?P5cj77iG)#hU/=b8@@!DaG+bqCs>RL6A51BM(3Va[b+9dg(`TWh/Qc7b+E:pLbWGG8uTUuYUShhK*>Hn8?S^.[MTIE&AuRl#Lm\sO'>NOR'[NTV:80BP7`FV12R_I;re'4qG972UB,juI>?4&#Op[V0
^A/).-Iou11Cp'IrUhV]U`d:1+7B@!fCgA(3b_!70dj0kppN'`+Y:%6XiOu<E9a/!D=J5(`^]\\PrS,Y=Wj/(mab^n/nqiFTZ_VfkV!=,E:A\1o3&R5]2Apj?Feo(/iCqU@-N#Jk!gRe:>2^2*Q:d7br.,q]RM'"l=/c_ZZfQ3][`mJ2Ka(3@&D%&Eph7h(LAX>i7J?Gpe#2Ug<SCe'*#9No/fK[K*]B]M<`@>&`=[;WnF59q_Xnr7o&,MIC5Mq>mEu8(u^[b^<:j8oe&Tk,#GlQ.TY0f&o^mh$IG5MlA"=`+~>
endstream
endobj
23 0 obj
@@ -351,51 +351,51 @@ endobj
xref
0 46
0000000000 65535 f
-0000009974 00000 n
-0000010053 00000 n
-0000010145 00000 n
+0000009996 00000 n
+0000010075 00000 n
+0000010167 00000 n
0000000015 00000 n
0000000071 00000 n
0000000813 00000 n
0000000933 00000 n
0000001000 00000 n
-0000010279 00000 n
+0000010301 00000 n
0000001135 00000 n
-0000010342 00000 n
+0000010364 00000 n
0000001272 00000 n
-0000010408 00000 n
+0000010430 00000 n
0000001409 00000 n
-0000010474 00000 n
+0000010496 00000 n
0000001546 00000 n
-0000010539 00000 n
+0000010561 00000 n
0000001682 00000 n
-0000010605 00000 n
+0000010627 00000 n
0000001819 00000 n
-0000010669 00000 n
+0000010691 00000 n
0000001955 00000 n
-0000004308 00000 n
-0000004431 00000 n
-0000004472 00000 n
-0000004650 00000 n
-0000004835 00000 n
-0000005035 00000 n
-0000006910 00000 n
-0000007018 00000 n
-0000007623 00000 n
-0000010735 00000 n
-0000007731 00000 n
-0000008157 00000 n
-0000008342 00000 n
-0000008506 00000 n
-0000008712 00000 n
-0000008934 00000 n
-0000009168 00000 n
-0000009312 00000 n
-0000009420 00000 n
-0000009530 00000 n
-0000009643 00000 n
-0000009759 00000 n
-0000009865 00000 n
+0000004330 00000 n
+0000004453 00000 n
+0000004494 00000 n
+0000004672 00000 n
+0000004857 00000 n
+0000005057 00000 n
+0000006932 00000 n
+0000007040 00000 n
+0000007645 00000 n
+0000010757 00000 n
+0000007753 00000 n
+0000008179 00000 n
+0000008364 00000 n
+0000008528 00000 n
+0000008734 00000 n
+0000008956 00000 n
+0000009190 00000 n
+0000009334 00000 n
+0000009442 00000 n
+0000009552 00000 n
+0000009665 00000 n
+0000009781 00000 n
+0000009887 00000 n
trailer
<<
/Size 46
@@ -403,5 +403,5 @@ trailer
/Info 4 0 R
>>
startxref
-10786
+10808
%%EOF
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html Tue Jun 28 02:51:00 2011
@@ -487,7 +487,7 @@ document.write("Last Published: " + docu
<pre class="code">
tickTime=2000
-dataDir=/var/zookeeper/
+dataDir=/var/lib/zookeeper/
clientPort=2181
initLimit=5
syncLimit=2
@@ -528,7 +528,7 @@ server.3=zoo3:2888:3888</pre>
<p>
-<span class="codefrag computeroutput">$ java -cp zookeeper.jar:lib/log4j-1.2.15.jar:conf \
+<span class="codefrag computeroutput">$ java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf \
org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg
</span>
</p>
@@ -565,7 +565,7 @@ server.3=zoo3:2888:3888</pre>
<p>
-<span class="codefrag computeroutput">$ java -cp zookeeper.jar:src/java/lib/log4j-1.2.15.jar:conf:src/java/lib/jline-0.9.94.jar \
+<span class="codefrag computeroutput">$ java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf:src/java/lib/jline-0.9.94.jar \
org.apache.zookeeper.ZooKeeperMain -server 127.0.0.1:2181</span>
</p>
@@ -888,7 +888,7 @@ server.3=zoo3:2888:3888</pre>
in the unlikely event a recent log has become corrupted). This
can be run as a cron job on the ZooKeeper server machines to
clean up the logs daily.</p>
-<pre class="code"> java -cp zookeeper.jar:log4j.jar:conf org.apache.zookeeper.server.PurgeTxnLog <dataDir> <snapDir> -n <count></pre>
+<pre class="code"> java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf org.apache.zookeeper.server.PurgeTxnLog <dataDir> <snapDir> -n <count></pre>
<a name="N10244"></a><a name="Debug+Log+Cleanup+%28log4j%29"></a>
<h4>Debug Log Cleanup (log4j)</h4>
<p>See the section on <a href="#sc_logging">logging</a> in this document. It is