You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by bu...@apache.org on 2013/01/18 13:13:03 UTC
svn commit: r847133 [2/3] - in /websites/staging/zookeeper/trunk/content: ./
bookkeeper/ bookkeeper/docs/r4.0.0/ bookkeeper/docs/r4.1.0/
bookkeeper/docs/trunk/ doc/trunk/
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigMetadata.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigMetadata.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigMetadata.html Fri Jan 18 12:13:00 2013
@@ -0,0 +1,291 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="/bookkeeper/css/site.css" rel="stylesheet" type="text/css">
+ <link href="/bookkeeper/css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - Hedwig Metadata Management</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/bookkeeper"><img src="/bookkeeper/images/bookkeeper_logo_blk.png"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="/bookkeeper/images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Metadata Management</h1>
+
+<p>There are two classes of metadata that need to be managed in Hedwig: one is the <i>list of available hubs</i>, which is used to track server availability (ZooKeeper is designed naturally for this); while the other is for data structures to track <i>topic states</i> and <i>subscription states</i>. This second class can be handled by any key/value store which provides ah <i><span class="caps">CAS </span>(Compare And Set)</i> operation. The metadata in this class are:</p>
+
+<ul>
+<li><code>Topic Ownership</code>: tracks which hub server is assigned to serve requests for a specific topic.</li>
+<li><code>Topic Persistence Info</code>: records what <i>bookkeeper ledgers</i> are used to store messages for a specific topic and their message id ranges.</li>
+<li><code>Subscription Data</code>: records the preferences and subscription state for a specific subscription (topic, subscriber).</li>
+</ul>
+
+<p>Each kind of metadata is handled by a specific metadata manager. They are <i>TopicOwnershipManager</i>, <i>TopicPersistenceManager</i> and <i>SubscriptionDataManager</i>.</p>
+
+<h2>Topic Ownership Management</h2>
+
+<p>There are two ways to management topic ownership. One is leveraging ZooKeeper's ephemeral znodes to record the topic's owner info as a child ephemeral znode under its topic znode. When a hub server, owning a specific topic, crashes, the ephemeral znode which signifies topic ownership will be deleted due to the loss of the zookeeper session. Other hubs can then be assigned the ownership of the topic. The other one is to leverage the <i><span class="caps">CAS</span></i> operation provided by key/value stores to do leader election. <i><span class="caps">CAS</span></i> doesn't require the underlying key/value store to provide functionality similar to ZooKeeper's ephemeral nodes. With <i><span class="caps">CAS</span></i> it is possible to guarantee that only one hub server gains the ownership for a specific topic, which is more scalable and generic solution.</p>
+
+<p>The implementation of a <i>TopicOwnershipManager</i> is required to implement following methods:</p>
+
+
+
+<pre><code>
+
+public void readOwnerInfo(ByteString topic, Callback<Versioned<HubInfo>> callback, Object ctx);
+
+public void writeOwnerInfo(ByteString topic, HubInfo owner, Version version,
+ Callback<Version> callback, Object ctx);
+
+public void deleteOwnerInfo(ByteString topic, Version version,
+ Callback<Void> callback, Object ctx);
+
+</code></pre>
+
+
+
+<ul>
+<li><code>readOwnerInfo</code>: Read the owner info from the underlying key/value store. The implementation should take the responsibility of deserializing the metadata into a <i>HubInfo</i> object identifying a hub server. Also, its current <i>version</i> needs to be returned for future updates. If there is no owner info found for a topic, null value is returned.</li>
+</ul>
+
+<ul>
+<li><code>writeOwnerInfo</code>: Write the owner info into the underlying key/value store with the given <i>version</i>. If the current <i>version</i> in underlying key/value store doesn't equal to the provided <i>version</i>, the write should be rejected with <i>BadVersionException</i>. The new <i>version</i> should be returned for a successful write. <i>NoTopicOwnerInfoException</i> is returned if no owner info found for a topic.</li>
+</ul>
+
+<ul>
+<li><code>deleteOwnerInfo</code>: Delete the owner info from key/value store with the given <i>version</i>. The owner info should be removed if the current <i>version</i> in key/value store is equal to the provided <i>version</i>. Otherwise, the deletion should be rejected with <i>BadVersionException</i>. <i>NoTopicOwnerInfoException</i> is returned if no owner info is found for the topic.</li>
+</ul>
+
+<h2>Topic Persistence Info Management</h2>
+
+<p>Similar as <i>TopicOwnershipManager</i>, an implementation of <i>TopicPersistenceManager</i> is required to implement <span class="caps">READ</span>/WRITE/DELETE interfaces as below:</p>
+
+
+
+<pre><code>
+public void readTopicPersistenceInfo(ByteString topic,
+ Callback<Versioned<LedgerRanges>> callback, Object ctx);
+
+public void writeTopicPersistenceInfo(ByteString topic, LedgerRanges ranges, Version version,
+ Callback<Version> callback, Object ctx);
+
+public void deleteTopicPersistenceInfo(ByteString topic, Version version,
+ Callback<Void> callback, Object ctx);
+</code></pre>
+
+
+
+<ul>
+<li><code>readTopicPersistenceInfo</code>: Read the persistence info from the underlying key/value store. The implementation should take the responsibility of deserializing the metadata into a <i>LedgerRanges</i> object includes the ledgers used to store messages. Also, its current <i>version</i> needs to be returned for future updates. If there is no persistence info found for a topic, a null value is returned.</li>
+</ul>
+
+<ul>
+<li><code>writeTopicPersistenceInfo</code>: Write the persistence info into the underlying key/value store with the given <i>version</i>. If the current <i>version</i> in the underlying key/value store doesn't equal the provided <i>version</i>, the write should be rejected with <i>BadVersionException</i>. The new <i>version</i> should be returned on a successful write. <i>NoTopicPersistenceInfoException</i> is returned if no persistence info is found for a topic.</li>
+</ul>
+
+<ul>
+<li><code>deleteTopicPersistenceInfo</code>: Delete the persistence info from the key/value store with the given <i>version</i>. The owner info should be removed if the current <i>version</i> in the key/value store equals the provided <i>version</i>. Otherwise, the deletion should be rejected with <i>BadVersionException</i>. <i>NoTopicPersistenceInfoException</i> is returned if no persistence info is found for a topic.</li>
+</ul>
+
+<h2>Subscription Data Management</h2>
+
+<p><i>SubscriptionDataManager</i> has similar <span class="caps">READ</span>/CREATE/WRITE/DELETE interfaces as other managers. Besides that, the implementation needs to implement <i><span class="caps">READ SUBSCRIPTIONS</span></i> interface, which is to fetch all the subscriptions for a given topic.</p>
+
+
+
+<pre><code>
+public void createSubscriptionData(ByteString topic, ByteString subscriberId, SubscriptionData data,
+ Callback<Version> callback, Object ctx);
+
+public boolean isPartialUpdateSupported();
+
+public void updateSubscriptionData(ByteString topic, ByteString subscriberId, SubscriptionData dataToUpdate,
+ Version version, Callback<Version> callback, Object ctx);
+
+public void replaceSubscriptionData(ByteString topic, ByteString subscriberId, SubscriptionData dataToReplace,
+ Version version, Callback<Version> callback, Object ctx);
+
+public void deleteSubscriptionData(ByteString topic, ByteString subscriberId, Version version,
+ Callback<Void> callback, Object ctx);
+
+public void readSubscriptionData(ByteString topic, ByteString subscriberId,
+ Callback<Versioned<SubscriptionData>> callback, Object ctx);
+
+public void readSubscriptions(ByteString topic, Callback<Map<ByteString, Versioned<SubscriptionData>>> cb,
+ Object ctx);
+</code></pre>
+
+
+
+<h3>Create/Update Subscriptions</h3>
+
+<p>The metadata for a subscription includes two parts, one is preferences and the other one is subscription state. <i>SubscriptionPreferences</i> tracks all the preferences for a subscriber (etc. Application could store its customized preferences for message filtering), while <i>SubscriptionState</i> is used internally to track the message consumption state for a given subscriber. These two kinds of metadata are quite different: <i>SubscriptionPreferences</i> is not updated<br />
+frequently while <i>SubscriptionState</i> is be updated frequently when messages are consumed. If the underlying key/value store supports independent field update for a given key (subscription), <i>SubscriptionPreferences</i> and <i>SubscriptionState</i> could be stored as two different fields for a given subscription. In this case <i>isPartialUpdateSupported</i> should return true. Otherwise, <i>isPartialUpdateSupported</i> should return false and the implementation should serialize/deserialize <i>SubscriptionData</i> as an opaque blob.</p>
+
+<ul>
+<li><code>createSubscriptionData</code>: Create a subscription entry for a given topic. The initial <i>version</i> would be returned for a success creation. <i>SubscriptionStateExistsException</i> is returned if the subscription entry already exists.</li>
+</ul>
+
+<ul>
+<li><code>updateSubscriptionData/replaceSubscriptionData</code>: Update/replace the subscription data in the underlying key/value store with the given <i>version</i>. If the current <i>version</i> in underlying key/value store doesn't equal to the provided <i>version</i>, the update should be rejected with <i>BadVersionException</i>. The new <i>version</i> should be returned for a successful write. <i>NoSubscriptionStateException</i> is returned if no subscription entry is found for a subscription (topic, subscriber).</li>
+</ul>
+
+<h3>Read Subscriptions</h3>
+
+<ul>
+<li><code>readSubscriptionData</code>: Read the subscription data from the underlying key/value store. The implementation should take the responsibility of deserializing the metadata into a <i>SubscriptionData</i> object including its preferences and subscription state. Also, its current <i>version</i> needs to be returned for future updates. If there is no subscription data found for a subscription, a null value is returned.</li>
+</ul>
+
+<ul>
+<li><code>readSubscriptions</code>: Read all the subscription data from key/value store for a given topic. The implementation should take the responsibility of managing all subscription for a topic for efficient access. An empty map is returned if there are no subscriptions found for a given topic.</li>
+</ul>
+
+<h3>Delete Subscription</h3>
+
+<ul>
+<li><code>deleteSubscriptionData</code>: Delete the subscription data from the key/value store with given <i>version</i> for a specific subscription (topic, subscriber). The subscription info should be removed if current <i>version</i> in key/value store equals the provided <i>version</i>. Otherwise, the deletion should be rejected with <i>BadVersionException</i>. <i>NoSubscriptionStateException</i> is returned if no subscription data is found for a subscription (topic, subscriber).</li>
+</ul>
+
+<h1>How to choose a key/value store for Hedwig.</h1>
+
+<p>From the interface, several requirements needs to meet before picking up a key/value store for Hedwig:</p>
+
+<ul>
+<li><code>CAS</code>: The ability to do strict updates according to specific condition, i.e. a specific version (ZooKeeper) and same content (HBase).</li>
+<li><code>Optimized for Writes</code>: The metadata access pattern for Hedwig is read first and continuous updates.</li>
+<li><code>Optimized for retrieving all subscriptions for a topic</code>: Either hierarchical structures to maintain such relationships (ZooKeeper), or ordered key/value storage to cluster the subscription for a topic together, would provide efficient subscription data management.</li>
+</ul>
+
+<p><i>ZooKeeper</i> is the default implementation for Hedwig metadata management, which holds data in memory and provides filesystem-like namespace, meeting the above requirements. <i>ZooKeeper</i> is suitable for most Hedwig usecases. However, if your application needs to manage millions of topics/subscriptions, a more scalable solution would be <i>HBase</i>, which also meet the above requirements.</p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+
+<h3>Documentation</h3>
+
+<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
+<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
+<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigParams.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigParams.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigParams.html Fri Jan 18 12:13:00 2013
@@ -0,0 +1,204 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="/bookkeeper/css/site.css" rel="stylesheet" type="text/css">
+ <link href="/bookkeeper/css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - </title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/bookkeeper"><img src="/bookkeeper/images/bookkeeper_logo_blk.png"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="/bookkeeper/images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <p> <br />
+h1. Hedwig configuration parameters <br />
+ <br />
+This page contains detailed information about configuration parameters used for Hubs, Regions, ZooKeeper, and BookKeeper.<br />
+ <br />
+h2. Hedwig server configuration parameters</p>
+
+<p>Please also refer to the configuration file that comes with the distribution: <em>hedwig-server/conf/hw_server.conf</em>. </p>
+
+<h3>Region related parameters</h3>
+
+<table><tr><td><code>region</code></td><td>Region identifier. Default is "standalone".</td></tr><tr><td><code>regions</code></td><td>List of region identifiers, space separated. Default is empty.</td></tr><tr><td><code>inter_region_ssl_enabled (deprecated)</code></td><td>Enables <span class="caps">SSL </span>across regions. Default is false. <strong>Since this parameter has been deprecated, use <i>ssl_enabled</i> in <em>hedwig-server/conf/hw_region_client.conf</em> to enable <span class="caps">SSL </span>across regions instead.</strong></td></tr><tr><td><code>retry_remote_subscribe_thread_run_interval</code></td><td>This parameter is used to determine how often we run a thread to retry those failed remote subscriptions in asynchronous mode (in milliseconds). Default is 2 minutes.</td></tr></table>
+
+<h3>Hub server parameters</h3>
+
+<table><tr><td><code>standalone</code></td><td>Sets the hub server to run in standalone mode (no regions). Default is false.</td></tr><tr><td><code>server_port</code></td><td>Sets the server port that receives client connections. Default is 4080.</td></tr><tr><td><code>ssl_enabled</code></td><td>Enables <span class="caps">SSL.</span> Default is false.</td></tr><tr><td><code>ssl_server_port</code></td><td>Sets the server port for <span class="caps">SSL </span>connections. Default is 9876.</td></tr><tr><td><code>password</code></td><td>Password used for pkcs12 certificate.. Default is the empty string.</td></tr><tr><td><code>cert_name</code></td><td>Sets the name of the <span class="caps">SSL </span>certificate if available as a resource. Default is the null string.</td></tr><tr><td><code>cert_path</code></td><td>Sets the path to the <span class="caps">SSL </span>certificate if it is available as a file. Default is the null string.</td></tr></table>
+
+<h3>Read-ahead cache parameters</h3>
+
+<table><tr><td><code>readahead_enabled</code></td><td>Enables read-ahead. Enabled by default.</td></tr><tr><td><code>readahead_count</code></td><td>Number of messages to read ahead. Default is 10.</td></tr><tr><td><code>readahead_size</code></td><td>Maximum number of bytes to read during a scan. Default is 4 megabytes.</td></tr></table>
+
+<blockquote><p>Upon a range scan request for a given topic, two hints are provided as to when scanning should stop: the number of messages scanned and the total size of messages scanned. Scanning stops whenever one of these limits is exceeded.</p></blockquote>
+
+<table><tr><td><code>cache_size</code></td><td>Sets the size of the read-ahead cache. Default is the smallest of 2G or half the heap size.</td></tr><tr><td><code>cache_entry_ttl</code></td><td>Sets <span class="caps">TTL </span>for cache entries. Each time adding new entry into the cache, those expired cache entries would be discarded. If the value is set to zero or less than zero, cache entry will not be evicted until the cache is fullfilled or the messages are already consumed. Default is 0.</td></tr><tr><td><code>scan_backoff_ms</code></td><td>The backoff time (in milliseconds) to retry scans after failures. Default value is 1s (1000ms). Default is 1s.</td></tr><tr><td><code>num_readahead_cache_threads</code></td><td>Sets the number of threads to be used for the read-ahead mechanism. Default is the number of cores as returned with a call to <code>Runtime.getRuntime().availableProcessors()</code>.</td></tr></table>
+
+<h3>Publish and subscription parameters </h3>
+
+<table><tr><td><code>max_message_size</code></td><td>Sets the maximum message size. Default is 1.2 megabytes.</td></tr><tr><td><code>default_message_window_size</code></td><td>This parameter is used for setting the default maximum number of messages that can be delivered to a subscriber without being consumed. We pause delivery to a subscriber when reaching the window size. Default is unlimited (0).</td></tr><tr><td><code>consume_interval</code></td><td>Sets the number of messages consumed before persisting information about consumed messages. A value greater than one avoids persisting information about consumed messages upon every consumed message. Default is 50.</td></tr><tr><td><code>retention_secs</code></td><td>the interval to release a topic. If this parameter is greater than zero, then schedule a task to release an owned topic. Default is 0 (never released).</td><td><code>messages_consumed_thread_run_interval</code></td><td>Time interval (in milliseconds) to run messa
ges consumed timer task to<br />
+delete those consumed ledgers in BookKeeper. Default is 1 minute (60,000 ms).</td></tr></table>
+
+
+<h3>ZooKeeper parameters<br />
+ <br />
+| <code>zk_host</code> | Sets the ZooKeeper list of servers. Default is localhost:2181. |<br />
+| <code>zk_timeout</code> | Sets the ZooKeeper session timeout. Default is 2s. |</h3>
+
+<h3>BookKeeper parameters</h3>
+
+<table><tr><td><code>bk_ensemble_size</code></td><td>Sets the ensemble size. Default is 3.</td></tr><tr><td><code>bk_write_quorum_size</code></td><td>Sets the write quorum size. Default is 2.</td></tr><tr><td><code>bk_ack_quorum_size</code></td><td>Sets the ack quorum size. Default is 2.</td></tr></table>
+
+<blockquote><p>Note that the ack quorum size must be equal or smaller than the write quorum size.</p></blockquote>
+
+<table><tr><td><code>max_entries_per_ledger</code></td><td>Maximum number of entries before we roll a ledger. Default is unlimited (0).</td></tr></table>
+
+<h3>Metadata parameters</h3>
+
+<table><tr><td><code>zk_prefix</code></td><td>Sets the ZooKeeper path prefix. Default is <em>/hedwig</em>.</td></tr><tr><td><code>metadata_manager_based_topic_manager_enabled</code></td><td>Enables the use of a metadata manager for topic management. Default is false.</td></tr><tr><td><code>metadata_manager_factory_class</code></td><td>Sets the default factory for the metadata manager. Default is null.</td></tr></table>
+
+<h2>Region manager configuration parameters</h2>
+
+<p>Please also refer to the configuration file that comes with the distribution: <em>hedwig-server/conf/hw_region_client.conf</em>.</p>
+
+<p>| <code>ssl_enabled</code> | This parameter is a boolean flag indicating if communication with the server should be done via <span class="caps">SSL </span>for encryption. The Hedwig server hubs also need to be <span class="caps">SSL </span>enabled for this to work. Default value is false. |<br />
+| <code>max_message_size</code> | Sets the maximum message size in bytes. The default value is 2 MB (2097152). |<br />
+| <code>max_server_redirects</code> | Sets the maximum number of redirects we permit before signaling an error. Default value is 2. |<br />
+| <code>auto_send_consume_message_enabled</code> | A flag indicating whether the client library should automatically send consume messages to the server. Default value is true. |<br />
+| <code>consumed_messages_buffer_size</code> | Sets the number of messages we buffer before sending a consume message to the server. Default value is 5. |<br />
+| <code>max_outstanding_messages</code> | Support for client side throttling, sets the maximum number of outstanding messages. Default value is 10. |<br />
+| <code>server_ack_response_timeout</code> | Sets the timeout (in milliseconds) before we error out any existing requests. Default value is 30s (30,000). |<br />
+ </p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+
+<h3>Documentation</h3>
+
+<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
+<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
+<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html Fri Jan 18 12:13:00 2013
@@ -136,6 +136,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html Fri Jan 18 12:13:00 2013
@@ -64,6 +64,7 @@
<li><a href="./bookkeeperInternals.html">BookKeeper Internals</a></li>
<li><a href="./bookieRecovery.html">Bookie Recovery</a></li>
<li><a href="./bookkeeperStream.html">Using BookKeeper stream library</a></li>
+<li><a href="./bookkeeperMetadata.html">BookKeeper Metadata Management</a></li>
</ul>
<h2>BookKeeper Admin & Ops</h2>
@@ -79,6 +80,9 @@
<li><a href="./hedwigBuild.html">Building Hedwig, or how to set up Hedwig</a></li>
<li><a href="./hedwigUser.html">User's Guide, or how to program against the Hedwig <span class="caps">API </span>and how to run it</a></li>
<li><a href="./hedwigDesign.html">Developer's Guide, or Hedwig internals and hacking details</a></li>
+<li><a href="./hedwigParams.html">Configuration parameters</a></li>
+<li><a href="./hedwigMessageFilter.html">Message Filtering</a></li>
+<li><a href="./hedwigMetadata.html">Hedwig Metadata Management</a></li>
</ul>
<h2>Hedwig Admin & Ops</h2>
@@ -87,6 +91,12 @@
<li><a href="./hedwigConsole.html">Hedwig Console</a></li>
<li><a href="./hedwigJMX.html">Hedwig <span class="caps">JMX</span></a></li>
</ul>
+
+<h1>Metastore documentation</h1>
+
+<ul>
+<li><a href="./metastore.textile">Metastore Interface</a></li>
+</ul>
</div>
</td>
<td valign="top">
@@ -120,6 +130,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/metastore.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/metastore.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/metastore.html Fri Jan 18 12:13:00 2013
@@ -0,0 +1,183 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="/bookkeeper/css/site.css" rel="stylesheet" type="text/css">
+ <link href="/bookkeeper/css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - Metastore Interface</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/bookkeeper"><img src="/bookkeeper/images/bookkeeper_logo_blk.png"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="/bookkeeper/images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Metastore Interface</h1>
+
+<p>Although Apache BookKeeper provides <a href="./bookkeeperMetadata.html">LedgerManager</a> and <a href="./hedwigMetadata.html">Hedwig Metadata Managers</a> for users to plugin different metadata storages for both BookKeeper and Hedwig, it is quite difficult to implement a correct and efficient manager version based on the knowledge for both projects. The <i>MetaStore</i> interface extracts the commonality of the metadata storage interfaces and is provided for users to focus on adapting the underlying storage itself w/o having to worry about the detailed logic for BookKeeper and Hedwig.</p>
+
+<h2>MetaStore</h2>
+
+<p>The <i>MetaStore</i> interface provide users with access to <i>MetastoreTable__s used for BookKeeper and Hedwig metadata management. There are two kinds of table defined in a <i>MetaStore</i>, <i>MetastoreTable</i> which provides basic <i><span class="caps">PUT</span></i>,__GET</i>,__REMOVE__,__SCAN__ operations and which does not assume any ordering requirements from the underlying storage; and <i>MetastoreScannableTable</i> which is derived from <i>MetastoreTable</i>, but <strong>does</strong> assume that data is stored in key order in the underlying storage.</p>
+
+<ul>
+<li><code>getName</code>: Return the name of the <i>MetaStore</i>.</li>
+<li><code>getVersion</code>: Return current <i>MetaStore</i> plugin version.</li>
+<li><code>init</code>: Initialize the <i>MetaStore</i> library with the given configuration and its version.</li>
+<li><code>close</code>: Close the <i>MetaStore</i>, freeing all resources. i.e. release all the open connections and occupied memory etc.</li>
+<li><code>createTable</code>: Create a table instance to access the data stored in it. A table name is given to locate the table. An <i>MetastoreTable</i> object is returned.</li>
+<li><code>createScannableTable</code>: Similar as <i>createTable</i>, but returns <i>MetastoreScannableTable</i> rather then <i>MetastoreTable</i> object. If the underlying table is not an ordered table, <i>MetastoreException</i> should be thrown.</li>
+</ul>
+
+<h2>MetaStore Table</h2>
+
+<p><i>MetastoreTable</i> is a basic unit in a <i>MetaStore</i>, which is used to handle different types of metadata, i.e. A <i>MetastoreTable</i> is used to store metadata for ledgers, while the other <i>MetastoreTable</i> is used to store metadata for topic persistence info. The interface for a <i>MetastoreTable</i> is quite simple:</p>
+
+<ul>
+<li><code>get</code>: Retrieve a entry by a given <i>key</i>. <i>OK</i> and its current version in metadata storage is returned when succeed. <i>NoKey</i> returned for a non-existent key. If <i>fields</i> are specified, return only the specified fields for the key.</li>
+<li><code>put</code>: Put the given <i>value</i> associated with <i>key</i> with given <i>version</i>. The value is only updated when the given <i>version</i> equals the current version in metadata storage. A new <i>version</i> should be returned when updated successfully. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when an update is attempted with a <i>version</i> which does not match the one in the metadata store.</li>
+<li><code>remove</code>: Remove the given <i>value</i> associated with <i>key</i>. The value is only removed when the given <i>version</i> equals its current version in metadata storage. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when remove is attempted with a <i>version</i> which does not match.</li>
+<li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table. The returned cursor doesn't need to guarantee any order and transaction.</li>
+</ul>
+
+<h2>MetaStore Scannable Table</h2>
+
+<p><i>MetastoreScannableTable</i> is identical to a <i>MetastoreTable</i> except that it provides an addition interface to iterate over entries in the table in key order.</p>
+
+<ul>
+<li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table between the key range of <i>firstKey</i> and <i>lastKey</i>.</li>
+</ul>
+
+<h2>How to organize your metadata.</h2>
+
+<p>Some metadata in Hedwig and BookKeeper does not need to be stored in the order of the ledger id or the topic. You could use kind of hash table to store metadata for them. These metadata are topic ownership and topic persistence info. Besides that, subscription state and ledger metadata must be stored in key order due to the current logic in Hedwig/BookKeeper.</p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+
+<h3>Documentation</h3>
+
+<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
+<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
+<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/index.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/index.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/index.html Fri Jan 18 12:13:00 2013
@@ -105,6 +105,7 @@ Learn more about Hedwig on the <a href="
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/irc.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/irc.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/irc.html Fri Jan 18 12:13:00 2013
@@ -91,6 +91,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/lists.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/lists.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/lists.html Fri Jan 18 12:13:00 2013
@@ -128,6 +128,7 @@ In order to post to the list, it is nece
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html Fri Jan 18 12:13:00 2013
@@ -103,6 +103,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/releases.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/releases.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/releases.html Fri Jan 18 12:13:00 2013
@@ -115,6 +115,7 @@ See <a href="./docs/r4.0.0/releaseNotes.
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/bookkeeper/svn.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/svn.html (original)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/svn.html Fri Jan 18 12:13:00 2013
@@ -105,6 +105,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.2.0">Release 4.2.0</a></li>
<li><a href="/bookkeeper/docs/r4.1.0">Release 4.1.0</a></li>
<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/index.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/index.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/index.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -330,36 +315,6 @@ document.write("Last Published: " + docu
</ul>
</li>
-
-
-<li>
-<strong>BookKeeper Documentation</strong>
-
-<p> BookKeeper is a highly-available system that implements high-performance write-ahead logging. It uses ZooKeeper for metadata,
- which is the main reason for being a ZooKeeper contrib.
- </p>
-
-<ul>
-
-<li>
-<a href="bookkeeperOverview.html">henn, what's it again?</a>
-</li>
-
-<li>
-<a href="bookkeeperStarted.html">Ok, now how do I try it out</a>
-</li>
-
-<li>
-<a href="bookkeeperProgrammer.html">Awesome, but how do I integrate it with my app?</a>
-</li>
-
-<li>
-<a href="bookkeeperStream.html">Can I stream bytes instead of entries?</a>
-</li>
-
-</ul>
-
-</li>
</ul>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/index.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/index.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/index.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -246,31 +231,35 @@ document.write("Last Published: " + docu
<ul>
<li>
<p>It takes as parameters:</p>
-
+
<ul>
-
+
<li>
<p>the address of the ZooKeeper service</p>
</li>
-
-<li>
-<p>then name of a znode - the one to be watched</p>
+
+<li>
+<p>the name of a znode - the one to be watched</p>
+</li>
+
+<li>
+<p>the name of a file to write the output to</p>
</li>
-
+
<li>
<p>an executable with arguments.</p>
</li>
</ul>
</li>
-
+
<li>
<p>It fetches the data associated with the znode and starts the executable.</p>
</li>
-
+
<li>
<p>If the znode changes, the client refetches the contents and restarts the executable.</p>
</li>
-
+
<li>
<p>If the znode disappears, the client kills the executable.</p>
</li>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/javaExample.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -290,40 +275,6 @@ document.write("Last Published: " + docu
<ul>
<li>
-<a>BookKeeper</a> ___________________ <em>docs</em>
-</li>
-<ul>
-
-<ul>
-<li>
-<a href="bookkeeperStarted.html">Getting started</a> ___________________ <em>bkStarted</em>
-</li>
-</ul>
-
-<ul>
-<li>
-<a href="bookkeeperOverview.html">Overview</a> ___________________ <em>bkOverview</em>
-</li>
-</ul>
-
-<ul>
-<li>
-<a href="bookkeeperConfig.html">Setup guide</a> ___________________ <em>bkProgrammer</em>
-</li>
-</ul>
-
-<ul>
-<li>
-<a href="bookkeeperProgrammer.html">Programmer's guide</a> ___________________ <em>bkProgrammer</em>
-</li>
-</ul>
-
-</ul>
-</ul>
-
-
-<ul>
-<li>
<a>Admin & Ops</a> ___________________ <em>docs</em>
</li>
<ul>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/linkmap.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/recipes.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/recipes.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/recipes.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<div class="menupagetitle">Recipes</div>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -949,7 +934,7 @@ document.write("Last Published: " + docu
<li>
-<p>Watch for changes on "ELECTION/guid-n_j", where j is the smallest
+<p>Watch for changes on "ELECTION/guid-n_j", where j is the largest
sequence number such that j < i and n_j is a znode in C;</p>
</li>
@@ -976,7 +961,7 @@ document.write("Last Published: " + docu
<li>
<p>Otherwise, watch for changes on "ELECTION/guid-n_j", where j is the
- smallest sequence number such that j < i and n_j is a znode in C;
+ largest sequence number such that j < i and n_j is a znode in C;
</p>
</li>
@@ -989,7 +974,7 @@ document.write("Last Published: " + docu
<p>Note that the znode having no preceding znode on the list of
children does not imply that the creator of this znode is aware that it is
- the current leader. Applications may consider creating a separate to znode
+ the current leader. Applications may consider creating a separate znode
to acknowledge that the leader has executed the leader procedure. </p>
</li>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/recipes.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/recipes.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/recipes.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/releasenotes.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html Fri Jan 18 12:13:00 2013
@@ -135,23 +135,8 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
-<div id="menu_1.3" class="menuitemgroup">
-<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_selected_1.4', 'skin/')" id="menu_selected_1.4Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
-<div id="menu_selected_1.4" class="selectedmenuitemgroup" style="display: block;">
+<div onclick="SwitchMenu('menu_selected_1.3', 'skin/')" id="menu_selected_1.3Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
+<div id="menu_selected_1.3" class="selectedmenuitemgroup" style="display: block;">
<div class="menupage">
<div class="menupagetitle">Administrator's Guide</div>
</div>
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -297,6 +282,9 @@ document.write("Last Published: " + docu
<a href="#Unsafe+Options">Unsafe Options</a>
</li>
<li>
+<a href="#Disabling+data+directory+autocreation">Disabling data directory autocreation</a>
+</li>
+<li>
<a href="#Communication+using+the+Netty+framework">Communication using the Netty framework</a>
</li>
</ul>
@@ -1460,6 +1448,54 @@ server.3=zoo3:2888:3888</pre>
</dd>
</dl>
+<a name="Disabling+data+directory+autocreation"></a>
+<h4>Disabling data directory autocreation</h4>
+<p>
+<strong>New in 3.5:</strong> The default
+ behavior of a ZooKeeper server is to automatically create the
+ data directory (specified in the configuration file) when
+ started if that directory does not already exist. This can be
+ inconvenient and even dangerous in some cases. Take the case
+ where a configuration change is made to a running server,
+ wherein the <strong>dataDir</strong> parameter
+ is accidentally changed. When the ZooKeeper server is
+ restarted it will create this non-existent directory and begin
+ serving - with an empty znode namespace. This scenario can
+ result in an effective "split brain" situation (i.e. data in
+ both the new invalid directory and the original valid data
+ store). As such is would be good to have an option to turn off
+ this autocreate behavior. In general for production
+ environments this should be done, unfortunately however the
+ default legacy behavior cannot be changed at this point and
+ therefore this must be done on a case by case basis. This is
+ left to users and to packagers of ZooKeeper distributions.
+ </p>
+<p>When running <strong>zkServer.sh</strong> autocreate can be disabled
+ by setting the environment variable <strong>ZOO_DATADIR_AUTOCREATE_DISABLE</strong> to 1.
+ When running ZooKeeper servers directly from class files this
+ can be accomplished by setting <strong>zookeeper.datadir.autocreate=false</strong> on
+ the java command line, i.e. <strong>-Dzookeeper.datadir.autocreate=false</strong>
+
+</p>
+<p>When this feature is disabled, and the ZooKeeper server
+ determines that the required directories do not exist it will
+ generate an error and refuse to start.
+ </p>
+<p>A new script <strong>zkServer-initialize.sh</strong> is provided to
+ support this new feature. If autocreate is disabled it is
+ necessary for the user to first install ZooKeeper, then create
+ the data directory (and potentially txnlog directory), and
+ then start the server. Otherwise as mentioned in the previous
+ paragraph the server will not start. Running <strong>zkServer-initialize.sh</strong> will create the
+ required directories, and optionally setup the myid file
+ (optional command line parameter). This script can be used
+ even if the autocreate feature itself is not used, and will
+ likely be of use to users as this (setup, including creation
+ of the myid file) has been an issue for users in the past.
+ Note that this script ensures the data directories exist only,
+ it does not create a config file, but rather requires a config
+ file to be available in order to execute.
+ </p>
<a name="Communication+using+the+Netty+framework"></a>
<h4>Communication using the Netty framework</h4>
<p>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperHierarchicalQuorums.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_selected_1.5', 'skin/')" id="menu_selected_1.5Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Contributor</div>
-<div id="menu_selected_1.5" class="selectedmenuitemgroup" style="display: block;">
+<div onclick="SwitchMenu('menu_selected_1.4', 'skin/')" id="menu_selected_1.4Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Contributor</div>
+<div id="menu_selected_1.4" class="selectedmenuitemgroup" style="display: block;">
<div class="menupage">
<div class="menupagetitle">ZooKeeper Internals</div>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.html Fri Jan 18 12:13:00 2013
@@ -135,23 +135,8 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
-<div id="menu_1.3" class="menuitemgroup">
-<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_selected_1.4', 'skin/')" id="menu_selected_1.4Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
-<div id="menu_selected_1.4" class="selectedmenuitemgroup" style="display: block;">
+<div onclick="SwitchMenu('menu_selected_1.3', 'skin/')" id="menu_selected_1.3Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
+<div id="menu_selected_1.3" class="selectedmenuitemgroup" style="display: block;">
<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperJMX.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.html Fri Jan 18 12:13:00 2013
@@ -135,23 +135,8 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
-<div id="menu_1.3" class="menuitemgroup">
-<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_selected_1.4', 'skin/')" id="menu_selected_1.4Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
-<div id="menu_selected_1.4" class="selectedmenuitemgroup" style="display: block;">
+<div onclick="SwitchMenu('menu_selected_1.3', 'skin/')" id="menu_selected_1.3Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Admin & Ops</div>
+<div id="menu_selected_1.3" class="selectedmenuitemgroup" style="display: block;">
<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<div class="menupagetitle">Observers Guide</div>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperObservers.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperOver.pdf Fri Jan 18 12:13:00 2013 differ
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.html (original)
+++ websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.html Fri Jan 18 12:13:00 2013
@@ -135,24 +135,9 @@ document.write("Last Published: " + docu
<a href="recipes.html">Recipes</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
+<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin & Ops</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
-<a href="bookkeeperStarted.html">Getting started</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperOverview.html">Overview</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperConfig.html">Setup guide</a>
-</div>
-<div class="menuitem">
-<a href="bookkeeperProgrammer.html">Programmer's guide</a>
-</div>
-</div>
-<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin & Ops</div>
-<div id="menu_1.4" class="menuitemgroup">
-<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
@@ -165,14 +150,14 @@ document.write("Last Published: " + docu
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
-<div id="menu_1.5" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
+<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
-<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
-<div id="menu_1.6" class="menuitemgroup">
+<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
+<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
</div>
@@ -952,6 +937,38 @@ document.write("Last Published: " + docu
a saved session id and password. One of the clients will reestablish the connection
and the second client will be disconnected (causing the pair to attempt to re-establish
it's connection/session indefinitely).</p>
+<p>
+<strong>Updating the list of servers</strong>. We allow a client to
+ update the connection string by providing a new comma separated list of host:port pairs,
+ each corresponding to a ZooKeeper server. The function invokes a probabilistic load-balancing
+ algorithm which may cause the client to disconnect from its current host with the goal
+ to achieve expected uniform number of connections per server in the new list.
+ In case the current host to which the client is connected is not in the new list
+ this call will always cause the connection to be dropped. Otherwise, the decision
+ is based on whether the number of servers has increased or decreased and by how much.
+ </p>
+<p>
+ For example, if the previous connection string contained 3 hosts and now the list contains
+ these 3 hosts and 2 more hosts, 40% of clients connected to each of the 3 hosts will
+ move to one of the new hosts in order to balance the load. The algorithm will cause the client
+ to drop its connection to the current host to which it is connected with probability 0.4 and in this
+ case cause the client to connect to one of the 2 new hosts, chosen at random.
+ </p>
+<p>
+ Another example -- suppose we have 5 hosts and now update the list to remove 2 of the hosts,
+ the clients connected to the 3 remaining hosts will stay connected, whereas all clients connected
+ to the 2 removed hosts will need to move to one of the 3 hosts, chosen at random. If the connection
+ is dropped, the client moves to a special mode where he chooses a new server to connect to using the
+ probabilistic algorithm, and not just round robin.
+ </p>
+<p>
+ In the first example, each client decides to disconnect with probability 0.4 but once the decision is
+ made, it will try to connect to a random new server and only if it cannot connect to any of the new
+ servers will it try to connect to the old ones. After finding a server, or trying all servers in the
+ new list and failing to connect, the client moves back to the normal mode of operation where it picks
+ an arbitrary server from the connectString and attempt to connect to it. If that fails, is will continue
+ trying different random servers in round robin. (see above the algorithm used to initially choose a server)
+ </p>
</div>
Modified: websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.pdf
==============================================================================
Files websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.pdf (original) and websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperProgrammers.pdf Fri Jan 18 12:13:00 2013 differ