You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by iv...@apache.org on 2011/12/07 15:24:00 UTC
svn commit: r799874 [2/2] - in /websites/production/zookeeper: ./
content/bookkeeper/ content/bookkeeper/docs/r4.0.0/
content/bookkeeper/docs/r4.0.0/apidocs/
content/bookkeeper/docs/r4.0.0/apidocs/org/
content/bookkeeper/docs/r4.0.0/apidocs/org/apache/...
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperConfig.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperConfig.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperConfig.html Wed Dec 7 14:23:53 2011
@@ -57,125 +57,45 @@
<p>This document contains information about deploying, administering and mantaining BookKeeper. It also discusses best practices and common problems. </p>
-<p>As BookKeeper is still a prototype, this article is likely to change significantly over time. </p>
+<h1>Running a BookKeeper instance</h1>
-<h1>System requirements</h1>
+<h2>System requirements</h2>
-<p>A typical BookKeeper installation comprises a set of bookies and a set of ZooKeeper replicas. The exact number of bookies depends on the quorum mode, desired throughput, and number of clients using this installation simultaneously. The minimum number of bookies is three for self-verifying (stores a message authentication code along with each entry) and four for generic (does not store a message authentication codewith each entry), and there is no upper limit on the number of bookies. Increasing the number of bookies, in fact, enables higher throughput. </p>
+<p>A typical BookKeeper installation comprises a set of bookies and a set of ZooKeeper replicas. The exact number of bookies depends on the quorum mode, desired throughput, and number of clients using this installation simultaneously. The minimum number of bookies is three for self-verifying (stores a message authentication code along with each entry) and four for generic (does not store a message authentication code with each entry), and there is no upper limit on the number of bookies. Increasing the number of bookies will, in fact, enable higher throughput. </p>
-<p>For performance, we require each server to have at least two disks. It is possible to run a bookie with a single disk, but performance will be significantly lower in this case. Of course, it works with one disk, but performance is significantly lower. </p>
+<p>For performance, we require each server to have at least two disks. It is possible to run a bookie with a single disk, but performance will be significantly lower in this case.</p>
-<p>For ZooKeeper, there is no constraint with respect to the number of replicas. Having a single machine running ZooKeeper in standalone mode is sufficient for BookKeeper. For resilience purposes, it might be a good idea to run ZooKeeper in quorum mode with multiple servers. Please refer to the ZooKeeper documentation for detail on how to configure ZooKeeper with multiple replicas </p>
+<p>For ZooKeeper, there is no constraint with respect to the number of replicas. Having a single machine running ZooKeeper in standalone mode is sufficient for BookKeeper. For resilience purposes, it might be a good idea to run ZooKeeper in quorum mode with multiple servers. Please refer to the ZooKeeper documentation for detail on how to configure ZooKeeper with multiple replicas. </p>
-<h1>Running bookies</h1>
+<h2>Running bookies</h2>
<p>To run a bookie, we execute the following command: </p>
-<p> @ bookkeeper-server/bin/bookkeeper bookie</p>
+<p><code>bookkeeper-server/bin/bookkeeper bookie</code></p>
-<p>The configuration parameters, which can be set in bookkeeper-server/conf/bkenv.sh</p>
+<p>The configuration parameters can be set in bookkeeper-server/conf/bk_server.conf. </p>
-<ul>
-<li><span class="caps">BOOKIE</span>_PORT: Port number that the bookie listens on; </li>
-<li><span class="caps">BOOKIE</span>_ZOOKEEPER: Comma separated list of ZooKeeper servers with a hostname:port format; </li>
-<li><span class="caps">BOOKIE</span>_TXN_LOGDIR: Path for Log Device (stores bookie write-ahead log); </li>
-<li><span class="caps">BOOKIE</span>_DATA_DIR: Path for Ledger Device (stores ledger entries); </li>
-</ul>
-
-<p>Ideally, <code>/path_to_log_device/</code> and <code>/path_to_ledger_device/</code> are each in a different device. </p>
-
-<h1>Bookie Configuration</h1>
-
-<p>Bookie server stores its data in multiple ledger directories and its journal files in a journal directory. Ideally, storing journal files in a separate directory than data files would increase throughput and decrease latency.</p>
-
-<h2>Journal Configuration</h2>
-
-<p>Journal directory has one kind of files in it:</p>
-
-<ul>
-<li>{timestamp}.txn - holds transactions executed in the bookie server.</li>
-</ul>
-
-<p>Before persisting ledger index and data to disk, a bookie ensures that the transaction that represents the update is written to a journal in non-volatile storage. A new journal file is created using current timestamp when a bookie starts or an old journal file reaches its maximum size.</p>
+<p>The important parameters are:<br />
+ * <code>bookiePort</code>, Port number that the bookie listens on; <br />
+ * <code>zkServers</code>, Comma separated list of ZooKeeper servers with a hostname:port format; <br />
+ * <code>journalDir</code>, Path for Log Device (stores bookie write-ahead log); <br />
+ * <code>ledgerDir</code>, Path for Ledger Device (stores ledger entries); </p>
-<p>A bookie supports journal rolling to remove old journal files. In order to remove old journal files safely, bookie server records LastLogMark in Ledger Device, which indicates all updates (including index and data) before LastLogMark has been persisted to the Ledger Device.</p>
+<p>Ideally, <code>journalDir</code> and <code>ledgerDir</code> are each in a different device. See <a href="./bookkeeperConfigParams.html">BookKeeper Configuration Parameters</a> for a full list of configuration parameters.</p>
-<p>LastLogMark contains two parts:</p>
-
-<ul>
-<li>LastLogId - indicates which journal file the transaction persisted.</li>
-<li>LastLogPos - indicates the position the transaction persisted in LastLogId journal file.</li>
-</ul>
-
-<p>You may use following settings to further fine tune the behavior of your Bookie servers. Currently these configuration settings are set using Java system properties.</p>
-
-<ul>
-<li>journalMaxSizeMB<ul>
-<li>journal file size limitation. when a journal reaches this limitation, it will be closed and new journal file be created.</li>
-</ul></li>
-</ul>
-
-<ul>
-<li>journalMaxBackups<ul>
-<li>how many old journal files whose id is less than LastLogMark 's journal id.</li>
-</ul></li>
-</ul>
-<blockquote><p><span class="caps">NOTE</span>: keeping number of old journal files would be useful for manually recovery in special case.</p></blockquote>
+<h3>Logging</h3>
-<h1>ZooKeeper Metadata</h1>
+<p>BookKeeper uses <a href="http://www.slf4j.org">slf4j</a> for logging, with the log4j bindings enabled by default. To enable logging from a bookie, create a log4j.properties file and point the environment variable <span class="caps">BOOKIE</span>_LOG_CONF to the configuration file. The path to the log4j.properties file must be absolute.</p>
-<p>For BookKeeper, we require a ZooKeeper installation to store metadata, and to pass the list of ZooKeeper servers as parameter to the constructor of the BookKeeper class ( <code>org.apache.bookkeeper.client,BookKeeper</code> ). To setup ZooKeeper, please check the <a href="http://index.html">ZooKeeper documentation</a>. </p>
+<p><code>export BOOKIE_LOG_CONF=/tmp/log4j.properties</code><br />
+<code>bookkeeper-server/bin/bookkeeper bookie</code></p>
-<p>BookKeeper provides two mechnaisms to organize its metadata in ZooKeeper.</p>
+<h2>Setting up a test ensemble</h2>
-<ul>
-<li>FlatLedgerManager : All ledger metadata are placed as children in a single zookeeper path.</li>
-<li>HierarchicalLedgerManager : All ledger metadata are partitioned into 2-level znodes.</li>
-</ul>
-
-<h2>Flat Ledger Manager</h2>
-
-<p>All ledgers' metadata are put in a single zookeeper path, created using zookeeper sequential node, which can ensure uniqueness of ledger id. Each ledger node is prefixed with 'L'.</p>
-
-<p>Bookie server manages its owned active ledgers in a hash map. So it is easy for bookie server to find what ledgers are deleted from zookeeper and garbage collect them. And its garbage collection flow is described as below:</p>
-
-<ul>
-<li>Fetch all existing ledgers from zookeeper (*zkActiveLedgers*).</li>
-<li>Fetch all ledgers currently active within the Bookie (*bkActiveLedgers*).</li>
-<li>Loop over <strong>bkActiveLedgers</strong> to find those ledgers which do not exist in <strong>zkActiveLedgers</strong> and garbage collect them.</li>
-</ul>
-
-<h2>Hierarchical Ledger Manager</h2>
-
-<p>Hierarchical Ledger Manager first obtains a global unique id from ZooKeeper using a <span class="caps">EPHEMERAL</span>_SEQUENTIAL znode.</p>
-
-<p>Since ZooKeeper sequential counter has a format of %10d -- that is 10 digits with 0 (zero) padding, i.e. "<path>0000000001", HierarchicalLedgerManager splits the generated id into 3 parts :</p>
-
-<p> @ {level1 (2 digits)}{level2 (4 digits)}{level3 (4 digits)}</p>
-
-<p>These 3 parts are used to form the actual ledger node path used to store ledger metadata:</p>
-
-<p> @ {ledgers_root_path}/{level1}/{level2}/L{level3}</p>
-
-<p>E.g. Ledger 0000000001 is split into 3 parts 00, 0000, 00001, which is stored in znode /{ledgers_root_path}/00/0000/L0001. So each znode could have at most 10000 ledgers, which avoids the problem of the child list being larger than the maximum ZooKeeper packet size.</p>
-
-<p>Bookie server manages its active ledgers in a sorted map, which simplifies access to active ledgers in a particular (level1, level2) partition.</p>
-
-<p>Garbage collection in bookie server is processed node by node as follows:</p>
-
-<ul>
-<li>Fetching all level1 nodes, by calling zk#getChildren(ledgerRootPath).<ul>
-<li>For each level1 nodes, fetching their level2 nodes :</li>
-<li>For each partition (level1, level2) :<ul>
-<li>Fetch all existed ledgers from zookeeper belonging to partition (level1, level2) (*zkActiveLedgers*).</li>
-<li>Fetch all ledgers currently active in the bookie which belong to partition (level1, level2) (*bkActiveLedgers*).</li>
-<li>Loop over <strong>bkActiveLedgers</strong> to find those ledgers which do not exist in <strong>zkActiveLedgers</strong>, and garbage collect them.</li>
-</ul></li>
-</ul></li>
-</ul>
+<p>Sometimes it is useful to run a ensemble of bookies on your local machine for testing. We provide a utility for doing this. It will set up N bookies, and a zookeeper instance locally. The data on these bookies and of the zookeeper instance are not persisted over restarts, so obviously this should never be used in a production environment. To run a test ensemble of 10 bookies, do the following.</p>
-<blockquote><p><span class="caps">NOTE</span>: Hierarchical Ledger Manager is more suitable to manage large number of ledgers existed in BookKeeper.</p></blockquote>
+<p><code>bookkeeper-server/bin/bookkeeper localbookie 10</code></p>
</div>
</td>
<td valign="top">
@@ -194,6 +114,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -208,6 +129,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperOverview.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperOverview.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperOverview.html Wed Dec 7 14:23:53 2011
@@ -61,7 +61,7 @@
<p>BookKeeper is a replicated service to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers, can crash, corrupt data, discard data, but as long as there are enough bookies behaving correctly the service as a whole behaves correctly. </p>
-<p>The initial motivation for BookKeeper comes from the namenode of <span class="caps">HDFS.</span> Namenodes have to log operations in a reliable fashion so that recovery is possible in the case of crashes. We have found the applications for BookKeeper extend far beyond <span class="caps">HDFS, </span>however. Essentially, any application that requires an append storage can replace their implementations with BookKeeper. BookKeeper has the advantage of scaling throughput with the number of servers. </p>
+<p>The initial motivation for BookKeeper comes from the namenode of <span class="caps">HDFS.</span> Namenodes have to log operations in a reliable fashion so that recovery is possible in the case of crashes. We have found the applications for BookKeeper extend far beyond <span class="caps">HDFS, </span>however. Essentially, any application that requires an append storage can replace their implementations with BookKeeper. BookKeeper has the advantage of writing efficiently, replicating for fault tolerance, and scaling throughput with the number of servers through striping. </p>
<p>At a high level, a bookkeeper client receives entries from a client application and stores it to sets of bookies, and there are a few advantages in having such a service: </p>
@@ -148,7 +148,7 @@ p. A simple use of BooKeeper is to imple
<h1>Closing out ledgers</h1>
-<p>The process of closing out the ledger and finding the last ledger is difficult due to the durability guarantees of BookKeeper: </p>
+<p>The process of closing out the ledger and finding the last entry is difficult due to the durability guarantees of BookKeeper: </p>
<ul>
<li>If an entry has been successfully recorded, it must be readable. </li>
@@ -168,15 +168,17 @@ p. A simple use of BooKeeper is to imple
<li>Make sure that all entries between <em>LC</em> and <em>LR</em> are on a quorum of bookies; </li>
</ol>
-<h1>Data Management in Bookie Server</h1>
+<h1>Data Management in Bookies</h1>
+
+<p>This section gives an overview of how a bookie manages its ledger fragments. </p>
<h2>Basic</h2>
-<p>Bookie servers manage data in a log-structured way, which is implemented using three kind of files:</p>
+<p>Bookies manage data in a log-structured way, which is implemented using three kind of files:</p>
<ul>
-<li><em>Journal</em> : A journal file contains the BookKeeper transaction logs. Before any update takes place, a Bookie server ensures that a transaction describing the update is written to non-volatile storage. A new journal file is created once the Bookie server starts or the older journal file reaches the journal file size threshold.</li>
-<li><em>Entry Log</em> : An entry log file manages the written entries received from BookKeeper clients. Entries from different ledgers are aggregated and written sequentially, while their offsets are kept as pointers in <em>LedgerCache</em> for fast lookup. A new entry log file is created once the Bookie server starts or the older entry log file reaches the entry log size threshold. Old entry log files are removed by the <em>Garbage Collector Thread</em> once they are not associated with any active ledger.</li>
+<li><em>Journal</em> : A journal file contains the BookKeeper transaction logs. Before any update takes place, a bookie ensures that a transaction describing the update is written to non-volatile storage. A new journal file is created once the bookie starts or the older journal file reaches the journal file size threshold.</li>
+<li><em>Entry Log</em> : An entry log file manages the written entries received from BookKeeper clients. Entries from different ledgers are aggregated and written sequentially, while their offsets are kept as pointers in <em>LedgerCache</em> for fast lookup. A new entry log file is created once the bookie starts or the older entry log file reaches the entry log size threshold. Old entry log files are removed by the <em>Garbage Collector Thread</em> once they are not associated with any active ledger.</li>
<li><em>Index File</em> : An index file is created for each ledger, which comprises a header and several fixed-length index pages, recording the offsets of data stored in entry log files. </li>
</ul>
@@ -188,12 +190,12 @@ p. A simple use of BooKeeper is to imple
<h2>Add Entry</h2>
-<p>When a Bookie server receives entries from clients to be written, these entries will go through the following steps to be persisted to disk:</p>
+<p>When a bookie receives entries from clients to be written, these entries will go through the following steps to be persisted to disk:</p>
<ol>
<li>Append the entry in <em>Entry Log</em>, return its position { logId , offset } ;</li>
<li>Update the index of this entry in <em>Ledger Cache</em> ;</li>
-<li>Append a transaction of update of this entry in <em>Journal</em> ;</li>
+<li>Append a transaction corresponding to this entry update in <em>Journal</em> ;</li>
<li>Respond to BookKeeper client ;</li>
</ol>
@@ -217,15 +219,15 @@ p. A simple use of BooKeeper is to imple
<ol>
<li>Records a <em>LastLogMark</em> in memory. The <em>LastLogMark</em> contains two parts: first one is <em>txnLogId</em> (file id of a journal) and the second one is <em>txnLogPos</em> (offset in a journal). The <em>LastLogMark</em> indicates that those entries before it have been persisted to both index and entry log files.</li>
<li>Flushes dirty index pages from <em>LedgerCache</em> to index file, and flushes entry log files to ensure all buffered entries in entry log files are persisted to disk.<ol>
-<li>Ideally, a Bookie server just needs to flush index pages and entry log files that contains entries before <em>LastLogMark</em>. There is no such information in <em>LedgerCache</em> and <em>Entry Log</em> mapping to journal files, though. Consequently, the thread flushes <em>LedgerCache</em> and <em>Entry Log</em> entirely here, and may flush entries after the <em>LastLogMark</em>. Flushing more is not a problem, though, just redundant.</li>
+<li>Ideally, a bookie just needs to flush index pages and entry log files that contains entries before <em>LastLogMark</em>. There is no such information in <em>LedgerCache</em> and <em>Entry Log</em> mapping to journal files, though. Consequently, the thread flushes <em>LedgerCache</em> and <em>Entry Log</em> entirely here, and may flush entries after the <em>LastLogMark</em>. Flushing more is not a problem, though, just redundant.</li>
</ol>
</li>
<li>Persists <em>LastLogMark</em> to disk, which means entries added before <em>LastLogMark</em> whose entry data and index page were also persisted to disk. It is the time to safely remove journal files created earlier than <em>txnLogId</em>.<ol>
-<li>If a Bookie server has crashed before persisting <em>LastLogMark</em> to disk, it still has journal files containing entries for which index pages may not have been persisted. Consequently, when this Bookie server restarts, it inspects journal files to restore those entries; data isn't lost.</li>
+<li>If the bookie has crashed before persisting <em>LastLogMark</em> to disk, it still has journal files containing entries for which index pages may not have been persisted. Consequently, when this bookie restarts, it inspects journal files to restore those entries; data isn't lost.</li>
</ol></li>
</ol>
-<p>Using the above data flush mechanism, it is safe for the <em>Sync Thread</em> to skip data flushing when the Bookie server shuts down. However, in <em>Entry Logger</em>, it uses <em>BufferedChannel</em> to write entries in batches and there might be data buffered in <em>BufferedChannel</em> upon a shut down. Bookie server needs to ensure <em>Entry Logger</em> flushes its buffered data during shutting down. Otherwise, <em>Entry Log</em> files become corrupted with partial entries.</p>
+<p>Using the above data flush mechanism, it is safe for the <em>Sync Thread</em> to skip data flushing when the bookie shuts down. However, in <em>Entry Logger</em>, it uses <em>BufferedChannel</em> to write entries in batches and there might be data buffered in <em>BufferedChannel</em> upon a shut down. The bookie needs to ensure <em>Entry Logger</em> flushes its buffered data during shutting down. Otherwise, <em>Entry Log</em> files become corrupted with partial entries.</p>
<p>As described above, <em>EntryLogger#flush</em> is invoked in the following two cases:<br />
* in <em>Sync Thread</em> : used to ensure entries added before <em>LastLogMark</em> are persisted to disk.<br />
@@ -248,6 +250,7 @@ p. A simple use of BooKeeper is to imple
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -262,6 +265,7 @@ p. A simple use of BooKeeper is to imple
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html Wed Dec 7 14:23:53 2011
@@ -55,290 +55,121 @@
<div class="wiki-content">
<h1>Abstract</h1>
-<p>This guide contains detailed information about using BookKeeper for logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs.</p>
+<p>This guide contains detailed information about using BookKeeper for write ahead logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs. The main classes used by BookKeeper client are <a href="./apidocs/org/apache/bookkeeper/client/BookKeeper.html">BookKeeper</a> and <a href="./apidocs/org/apache/bookkeeper/client/LedgerHandle.html">LedgerHandle</a>. </p>
-<h1> Instantiating BookKeeper.</h1>
+<p>BookKeeper is the main client used to create, open and delete ledgers. A ledger is a log file in BookKeeper, which contains a sequence of entries. Only the client which creates a ledger can write to it. A LedgerHandle represents the ledger to the client, and allows the client to read and write entries. When the client is finished writing they can close the LedgerHandle. Once a ledger has been closed, all client who read from it are guaranteed to read the exact same entries in the exact same order. All methods of BookKeeper and LedgerHandle have synchronous and asynchronous versions. Internally the synchronous versions are implemented using the asynchronous.</p>
-<p>The first step to use BookKeeper is to instantiate a BookKeeper object: </p>
+<h1> Instantiating BookKeeper</h1>
-<p> <code>org.apache.bookkeeper.BookKeeper </code> </p>
+<p>To create a BookKeeper client, you need to create a configuration object and set the address of the ZooKeeper ensemble in use. For example, if you were using <code>zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181</code> as your ensemble, you would create the BookKeeper client as follows.</p>
-<p>There are three BookKeeper constructors: </p>
-<p> <code>public BookKeeper(String servers) throws KeeperException, IOException </code> </p>
-<p>where: </p>
-
-<ul>
-<li> <code>servers</code> is a comma-separated list of ZooKeeper servers. </li>
-</ul>
-
-
-<p> <code>public BookKeeper(ZooKeeper zk) throws InterruptedException, KeeperException </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>zk</code> is a ZooKeeper object. This constructor is useful when the application also using ZooKeeper and wants to have a single instance of ZooKeeper. </li>
-</ul>
-
-
-<p> <code>public BookKeeper(ZooKeeper zk, ClientSocketChannelFactory channelFactory) throws InterruptedException, KeeperException </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>zk</code> is a ZooKeeper object. This constructor is useful when the application also using ZooKeeper and wants to have a single instance of ZooKeeper. </li>
-<li> <code>channelFactory</code> is a netty channel object ( <code>org.jboss.netty.channel.socket</code> ). </li>
-</ul>
-
-
-<h1> Creating a ledger. </h1>
-
-<p>Before writing entries to BookKeeper, it is necessary to create a ledger. With the current BookKeeper <span class="caps">API, </span>it is possible to create a ledger both synchronously or asynchronously. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
-
-<p> <em>Synchronous call:</em> </p>
-
-<p> <code>public LedgerHandle createLedger(int ensSize, int qSize, DigestType type, byte passwd[]) throws KeeperException, InterruptedException, IOException, BKException </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>ensSize</code> is the number of bookies (ensemble size); </li>
-<li> <code>qSize</code> is the write quorum size; </li>
-<li> <code>type</code> is the type of digest used with entries: either <span class="caps">MAC </span>or <span class="caps">CRC32. </span></li>
-<li> <code>passwd</code> is a password that authorizes the client to write to the ledger being created. </li>
-</ul>
-
-
-<p>All further operations on a ledger are invoked through the <code>LedgerHandle</code> object returned. </p>
-
-<p>As a convenience, we provide a <code>createLedger</code> with default parameters (3,2,VERIFIABLE), and the only two input parameters it requires are a digest type and a password. </p>
-
-<p> <em>Asynchronous call:</em> </p>
-
-<p> <code>public void asyncCreateLedger(int ensSize, int qSize, DigestType type, byte passwd[], CreateCallback cb, Object ctx ) </code> </p>
-
-<p>The parameters are the same of the synchronous version, with the exception of <code>cb</code> and <code>ctx</code> . <code>CreateCallback</code> is an interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>createComplete</code> that has the following signature: </p>
-
-<p> <code>void createComplete(int rc, LedgerHandle lh, Object ctx); </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKException</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>ctx</code> is a control object for accountability purposes. It can be essentially any object the application is happy with. </li>
-</ul>
-
-
-<p>The <code>ctx</code> object passed as a parameter to the call to create a ledger is the one same returned in the callback. </p>
-
-<h1> Adding entries to a ledger. </h1>
-
-<p>Once we have a ledger handle <code>lh</code> obtained through a call to create a ledger, we can start writing entries. As with creating ledgers, we can write both synchronously and asynchronously. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
-
-<p> <em>Synchronous call:</em> </p>
-
-<p> <code>public long addEntry(byte[] data) throws InterruptedException </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>data</code> is a byte array; </li>
-</ul>
+<pre><code>
+ClientConfiguration conf = new ClientConfiguration();
+conf.setZkServers("zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181");
+BookKeeper client = new BookKeeper(conf);
+</code></pre>
-<p>A call to <code>addEntry</code> returns the status of the operation (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </p>
-<p> <em>Asynchronous call:</em> </p>
-<p> <code>public void asyncAddEntry(byte[] data, AddCallback cb, Object ctx) </code> </p>
+<p>It is important to close the client once you are finished working with it. The set calls on ClientConfiguration are chainable, so instead of putting a set* call on a new line as above, it is possible to make a number of calls on the one line. For example;</p>
-<p>It also takes a byte array as the sequence of bytes to be stored as an entry. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>AddCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>addComplete</code> that has the following signature: </p>
-<p> <code>void addComplete(int rc, LedgerHandle lh, long entryId, Object ctx); </code> </p>
-<p>where: </p>
+<pre><code>
+ClientConfiguration conf = new ClientConfiguration().setZkServers("localhost:2181").setZkTimeout(5000);
+</code></pre>
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>entryId</code> is the identifier of entry associated with this request; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. It can be any object the application is happy with. </li>
-</ul>
-<h1> Closing a ledger. </h1>
+<p>There is also a useful shortcut constructor which allows you to pass the zookeeper ensemble string directly to BookKeeper.</p>
-<p>Once a client is done writing, it closes the ledger. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
-<p> <em>Synchronous close:</em> </p>
+<pre><code>
+BookKeeper client = new BookKeeper("localhost:2181");
+</code></pre>
-<p> <code>public void close() throws InterruptedException </code> </p>
-<p>It takes no input parameters. </p>
-<p> <em>Asynchronous close:</em> </p>
+<p>See <a href="./apidocs/org/apache/bookkeeper/client/BookKeeper.html">BookKeeper</a> for the full api.</p>
-<p> <code>public void asyncClose(CloseCallback cb, Object ctx) throws InterruptedException </code> </p>
-<p>It takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>CloseCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>closeComplete</code> that has the following signature: </p>
+<h1> Creating a ledger</h1>
-<p> <code>void closeComplete(int rc, LedgerHandle lh, Object ctx) </code> </p>
+<p>Before writing entries to BookKeeper, it is necessary to create a ledger. Before creating the ledger you must decide the ensemble size and the quorum size. </p>
-<p>where: </p>
+<p>The ensemble size is the number of Bookies over which entries will be striped. The quorum size is the number of bookies which an entry will be written to. Striping is done in a round robin fashion. For example, if you have an ensemble size of 3 (consisting of bk1, bk2 & bk3), and a quorum of 2, entry 1 will be written to bk1 & bk2, entry 2 will be written to bk2 & bk3, entry 3 will be written to bk3 & bk1 and so on.</p>
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
+<p>Ledgers are also created with a digest type and password. The digest type is used to generate a checksum so that when reading entries we can ensure that the content is the same as what was written. The password is used as an access control mechanism.</p>
+<p>To create a ledger, with ensemble size 3, quorum size 2, using a <span class="caps">CRC </span>to checksum and "foobar" as the password, do the following:</p>
-<h1> Opening a ledger. </h1>
-<p>To read from a ledger, a client must open it first. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
-<p> <em>Synchronous open:</em> </p>
+<pre><code>
+LedgerHandle lh = client.createLedger(3, 2, DigestType.CRC32, "foobar");
+</code></pre>
-<p> <code>public LedgerHandle openLedger(long lId, DigestType type, byte passwd[]) throws InterruptedException, BKException </code> </p>
-<ul>
-<li> <code>lId</code> is the ledger identifier; </li>
-<li> <code>type</code> is the type of digest used with entries: either <span class="caps">MAC </span>or <span class="caps">CRC32. </span></li>
-<li> <code>passwd</code> is a password to access the ledger (used only in the case of <code>VERIFIABLE</code> ledgers); </li>
-</ul>
+<p>You can now write to this ledger handle. As you probably plan to read the ledger at some stage, now is a good time to store the id of the ledger somewhere. The ledger id is a long, and can be obtained with <code>lh.getId()</code>.</p>
-<p> <em>Asynchronous open:</em> </p>
+<h1> Adding entries to a ledger</h1>
-<p> <code>public void asyncOpenLedger(long lId, DigestType type, byte passwd[], OpenCallback cb, Object ctx) </code> </p>
+<p>Once you have obtained a ledger handle, you can start adding entries to it. Entries are simply arrays of bytes. As such, adding entries to the ledger is rather simple.</p>
-<p>It also takes a a ledger identifier and a password. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>OpenCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>openComplete</code> that has the following signature: </p>
-<p> <code>public void openComplete(int rc, LedgerHandle lh, Object ctx) </code> </p>
-<p>where: </p>
+<pre><code>
+lh.addEntry("Hello World!".getBytes());
+</code></pre>
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
-<p>The two calls above to open a ledger recover the ledger if it has not been closed properly before it reads entries from it. It is also possible to open a ledger and read from it without triggering the recovery process with the following methods of <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
-<p> <em>Synchronous open:</em> </p>
+<h1> Closing a ledger</h1>
-<p> <code>public LedgerHandle openLedgerNoRecovery(long lId, DigestType type, byte passwd[]) throws InterruptedException, BKException </code> </p>
+<p>Once a client is done writing, it can closes the ledger. Closing the ledger is a very important step in BookKeeper, as once a ledger is closed, all reading clients are guaranteed to read the same sequence of entries in the same order. Closing takes no parameters. </p>
-<ul>
-<li> <code>lId</code> is the ledger identifier; </li>
-<li> <code>type</code> is the type of digest used with entries: either <span class="caps">MAC </span>or <span class="caps">CRC32. </span></li>
-<li> <code>passwd</code> is a password to access the ledger (used only in the case of <code>VERIFIABLE</code> ledgers); </li>
-</ul>
-<p> <em>Asynchronous open:</em> </p>
+<pre><code>
+lh.close();
+</code></pre>
-<p> <code>public void asyncOpenLedgerNoRecovery(long lId, DigestType type, byte passwd[], OpenCallback cb, Object ctx) </code> </p>
-<p>It also takes a a ledger identifier and a password. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>OpenCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>openComplete</code> that has the following signature: </p>
-<p> <code>public void openComplete(int rc, LedgerHandle lh, Object ctx) </code> </p>
+<h1> Opening a ledger</h1>
-<p>where: </p>
+<p>To read from a ledger, a client must open it first. To open a ledger you must know its <span class="caps">ID, </span>which digest type was used when creating it, and its password. To open the ledger we created above, assuming it has ID 1;</p>
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
-<h1> Reading from ledger </h1>
-<p>Read calls may request one or more consecutive entries. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
+<pre><code>
+LedgerHandle lh2 = client.openLedger(1, DigestType.CRC32, "foobar");
+</code></pre>
-<p> <em>Synchronous read:</em> </p>
-<p> <code>public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry) throws InterruptedException, BKException </code> </p>
-<ul>
-<li> <code>firstEntry</code> is the identifier of the first entry in the sequence of entries to read; </li>
-<li> <code>lastEntry</code> is the identifier of the last entry in the sequence of entries to read. </li>
-</ul>
-
-
-<p> <em>Asynchronous read:</em> </p>
-
-<p> <code>public void asyncReadEntries(long firstEntry, long lastEntry, ReadCallback cb, Object ctx) throws BKException, InterruptedException </code> </p>
-
-<p>It also takes a first and a last entry identifiers. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>ReadCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>readComplete</code> that has the following signature: </p>
-
-<p> <code>void readComplete(int rc, LedgerHandle lh, Enumeration<LedgerEntry> seq, Object ctx) </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
-<li> <code>seq</code> is a <code>Enumeration<LedgerEntry></code> object to containing the list of entries requested; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
+<p>You can now read entries from the ledger. Any attempt to write to this handle will throw an exception.</p>
+<blockquote><p><span class="caps">NOTE</span>: Opening a ledger, which another client already has open for writing will prevent that client from writing any new entries to it. If you do not wish this to happen, you should use the openLedgerNoRecovery method. However, keep in mind that without recovery, you lose the guarantees of what entries are in the ledger. You should only use openLedgerNoRecovery if you know what you are doing.</p></blockquote>
-<h1> Deleting a ledger </h1>
+<h1>Reading entries from a ledger</h1>
-<p>Once a client is done with a ledger and is sure that nobody will ever need to read from it again, they can delete the ledger. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
+<p>Now that you have an open ledger, you can read entries from it. You can use <code>getLastAddConfirmed</code> to get the id of the last entry in the ledger.</p>
-<p> <em>Synchronous delete:</em> </p>
-<p> <code>public void deleteLedger(long lId) throws InterruptedException, BKException </code> </p>
-<ul>
-<li> <code>lId</code> is the ledger identifier; </li>
-</ul>
-
-
-<p> <em>Asynchronous delete:</em> </p>
-
-<p> <code>public void asyncDeleteLedger(long lId, DeleteCallback cb, Object ctx) </code> </p>
-
-<p>It takes a ledger identifier. Additionally, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>DeleteCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>deleteComplete</code> that has the following signature: </p>
-
-<p> <code>void deleteComplete(int rc, Object ctx) </code> </p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
-
-<h1> Reading the last confirmed entry from a ledger </h1>
-
-<p>When reading from an open ledger (opening without recovery enables it), it is often necessary to read the last confirmed hint that the writer client writes along with every entry. The general idea is to obtain the hint from a quorum and return the largest value. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
-
-<p> <em>Synchronous read:</em> </p>
-
-<p> <code>public long readLastConfirmed() </code> </p>
-
-<p> <em>Asynchronous read:</em> </p>
-
-<p> <code>public void asyncReadLastConfirmed(ReadLastConfirmedComplete cb, Object ctx) </code> <br />
- <br />
-p. It takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>ReadLastConfirmedCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>readLastConfirmedComplete</code> that has the following signature: </p>
-
-<p><code>public void readLastConfirmedComplete (int rc, long lastConfirmed, Object ctx) </code></p>
-
-<p>where: </p>
-
-<ul>
-<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
-<li> <code>lastConfirmed</code> is the maximum last confirmed hint received among a quorum of bookies; </li>
-<li> <code>ctx</code> is control object used for accountability purposes. </li>
-</ul>
+<pre><code>
+long lastEntry = lh2.getLastAddConfirmed();
+Enumeration<LedgerEntry> entries = lh2.readEntries(0, 9);
+while (entries.hasMoreElements()) {
+ byte[] bytes = entries.nextElement().getEntry();
+ System.out.println(new String(bytes));
+}
+</code></pre>
</div>
</td>
<td valign="top">
@@ -357,6 +188,7 @@ p. It takes a callback object <code>cb<
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -371,6 +203,7 @@ p. It takes a callback object <code>cb<
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStarted.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStarted.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStarted.html Wed Dec 7 14:23:53 2011
@@ -75,7 +75,7 @@
<p>To start a local bookkeeper ensemble with 5 bookies:</p>
-<p>@ bookkeeper-server/bin/bookkeeper localbookie 5</p>
+<p> <code>bookkeeper-server/bin/bookkeeper localbookie 5</code></p>
<h1>Setting up bookies</h1>
@@ -83,13 +83,13 @@
<p>For each bookie, we need to execute a command like the following: </p>
-<p> @ bookkeeper-server/bin/bookkeeper bookie</p>
+<p> <code>bookkeeper-server/bin/bookkeeper bookie</code></p>
-<p>This command will use the default directories for storing ledgers and the write ahead log, and will look for a zookeeper server on localhost:2181. To modify this you must edit bookkeeper-server/conf/bkenv.sh. <span class="caps">BOOKIE</span>_TXN_LOGDIR and <span class="caps">BOOKIE</span>_DATA_DIR should be configured to used different physical devices for best performance. <span class="caps">BOOKIE</span>_ZOOKEEPER should be a comma separated list of zookeeper servers.</p>
+<p>This command will use the default directories for storing ledgers and the write ahead log, and will look for a zookeeper server on localhost:2181. See the <a href="./bookkeeperConfig.html">Admin Guide</a> for more details.</p>
<p>To see the default values of these configuration variables, run:</p>
-<p> @ bookkeeper-server/bin/bookkeeper help</p>
+<p> <code>bookkeeper-server/bin/bookkeeper help</code></p>
<h1>Setting up ZooKeeper</h1>
@@ -106,15 +106,20 @@
<p>In the following excerpt of code, we: </p>
<ol>
+<li>Open a bookkeeper client;</li>
<li>Create a ledger; </li>
<li>Write to the ledger; </li>
<li>Close the ledger; </li>
<li>Open the same ledger for reading; </li>
<li>Read from the ledger; </li>
<li>Close the ledger again; </li>
+<li>Close the bookkeeper client.</li>
</ol>
+
+
<pre><code>
+BookKeeper bkc = new BookKeeper("localhost:2181");
LedgerHandle lh = bkc.createLedger(ledgerPassword);
ledgerId = lh.getId();
ByteBuffer entry = ByteBuffer.allocate(4);
@@ -140,7 +145,8 @@ while(ls.hasMoreElements()){
Integer retrEntry = result.getInt();
}
lh.close();
- </code></pre>
+bkc.close();
+</code></pre>
</div>
</td>
<td valign="top">
@@ -159,6 +165,7 @@ lh.close();
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -173,6 +180,7 @@ lh.close();
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStream.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStream.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/bookkeeperStream.html Wed Dec 7 14:23:53 2011
@@ -200,6 +200,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -214,6 +215,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/doc.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/doc.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/doc.html Wed Dec 7 14:23:53 2011
@@ -82,6 +82,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -96,6 +97,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigBuild.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigBuild.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigBuild.html Wed Dec 7 14:23:53 2011
@@ -60,8 +60,6 @@
<ul>
<li><span class="caps">JDK</span> 6: <a href="http://java.sun.com/">http://java.sun.com/</a>. Ensure <code>$JAVA_HOME</code> is correctly set.</li>
<li>Maven 2: <a href="http://maven.apache.org/">http://maven.apache.org/</a>.</li>
-<li>Protocol Buffers 2.3.0: <a href="http://protobuf.googlecode.com/">http://protobuf.googlecode.com/</a>.</li>
-<li>Zookeeper 3.4.0: <a href="http://zookeeper.apache.org/">http://zookeeper.apache.org/</a>.</li>
</ul>
<p>Hedwig has been tested on Windows <span class="caps">XP,</span> Linux 2.6, and OS X.</p>
@@ -82,8 +80,6 @@
<li>Go to Preferences > Team > <span class="caps">SVN.</span> For the <span class="caps">SVN </span>interface, choose "Pure Java".</li>
<li>Choose File > New > Project... > Maven > Checkout Maven Projects from <span class="caps">SCM.</span></li>
<li>For the <span class="caps">SCM URL </span>type, choose <span class="caps">SVN.</span> For the <span class="caps">URL, </span>enter <span class="caps">SVN URL.</span> Maven will automatically create a top-level Eclipse project for each of the 4 Maven modules (recommended). If you want fewer top-level projects, uncheck the option of having a project for each module (under Advanced).</li>
-<li>Right-click on the <code>protocol</code> project and choose Run As > Maven generate-sources. This will generate the Java and C++ code for Protocol Buffers.</li>
-<li>Refresh the workspace to pick up the generated code and add <code>hedwig/protocol/target/generated-sources/java</code> as a source folder. (6 & 7 should really be doable automatically, but I haven't figured out how.)</li>
</ol>
<p>You are now ready to run and debug the client and server code. See the User's Guide for instructions on running and usage.</p>
@@ -105,6 +101,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -119,6 +116,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigDesign.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigDesign.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigDesign.html Wed Dec 7 14:23:53 2011
@@ -80,9 +80,13 @@
<p>This summarizes the control flow through the system.</p>
-<p>publishhandler topicmgr.getowner (maybe) claim the topic, calling back into persmgr.acquiredtopic read /hedwig/standalone/topics/TOPIC (which should initially be empty) for each line, parse as "STARTSEQ" # <span class="caps">TODO </span>how is this written? ledger = bk.openledger(ledgerid) lastid = ledger.getlast if lastid > 0, lrs[startseq] = persmgr.ledger2lr[ledgerid] = new LedgerRange(ledger, ledgerid, startseq, startseq + lastid # <span class="caps">TODO </span>what are ledger ranges? create new ledger for topic # <span class="caps">TODO </span>read lr = new LedgerRange(ledger, ledgerid, lastid, -1) lrs[lastid] = lr persmgr.topic2ranges[topic] = lrs add region info to pub req and send that to persmgr.persistmessage entryid = persmgr.topic2ranges[topic].last.ledger.addentry(the pub'd data) update persmgr.topic2lastseq[topic]: .local = persmgr.ledger2lr[ledger id].startseq + entryid .regions = maxes of orig seq and incoming pub seq</p>
+<h2>publish</h2>
-<p>subscribehandler topicmgr.getowner... delivmgr.startservingsubscription(topic, endpoint, ishubsubscriber) delivmgr.endpoint2sub[endpoint] = new subscriber(lastseq = persmgr.getcurrentseqidfortopic(topic).local) delivmgr.topic2ptr2subs[topic][ptr].add(sub) sub.delivernextmessage sub.curseq = persmgr.getseqidafterskipping(topic, sub.lastseq, skip = 1) msg = persmgr.scansinglemessage(topic, seq = sub.curseq) if persmgr.topic2lastseq[topic].local >= seq lr = persmgr.topic2ranges[topic].floor(seq) return lr.ledger.read(first = last = seq - lr.startseq) if failed, then retry in 1 s endpoint.send(msg) movedeliveryptr delivmgr.topic2ptr2subs[topic][sub.lastseq].remove(sub) delivmgr.topic2ptr2subs[topic][sub.curseq].add(sub) previd = sub.lastseq, sub.lastseq = sub.curseq sub.delivernextmessage...</p>
+<p>Need to document</p>
+
+<h2>subscribe</h2>
+
+<p>Need to document</p>
<h1>ReadAhead Cache</h1>
@@ -131,6 +135,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -145,6 +150,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigUser.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigUser.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/hedwigUser.html Wed Dec 7 14:23:53 2011
@@ -87,7 +87,7 @@
<p>To start a Hedwig hub server:</p>
-<p> @ hedwig-server/bin/hedwig server</p>
+<p><code>hedwig-server/bin/hedwig server</code></p>
<p>Hedwig takes its configuration from hedwig-server/conf/hw_server.conf by default. To change location of the conf file, modify the <span class="caps">HEDWIG</span>_SERVER_CONF environment variable.</p>
@@ -95,13 +95,14 @@
<p>You can attach an Eclipse debugger (or any debugger) to a Java process running on a remote host, as long as it has been started with the appropriate <span class="caps">JVM </span>flags. (See the Building Hedwig document to set up your Eclipse environment.) To launch something using <code>bin/hedwig</code> with debugger attachment enabled, prefix the command with <code>HEDWIG_EXTRA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=5000</code>, e.g.:</p>
-<p>@ <span class="caps">HEDWIG</span>_EXTRA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=5000 hedwig-server/bin/hedwig server</p>
+<p><code>HEDWIG_EXTRA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=5000 hedwig-server/bin/hedwig server</code></p>
<h1>Logging</h1>
-<p>To control the logging in Hedwig server, set <span class="caps">HEDWIG</span>_LOG_CONF to the location of a log4j.properties file.</p>
+<p>Hedwig uses <a href="http://www.slf4j.org">slf4j</a> for logging, with the log4j bindings enabled by default. To enable logging from hedwig, create a log4j.properties file and point the environment variable <span class="caps">HEDWIG</span>_LOG_CONF to the file. The path to the log4j.properties file must be absolute.</p>
-<p>@ <span class="caps">HEDWIG</span>_LOG_CONF=../hw_testing/log4j.properties hedwig-server/bin/hedwig server</p>
+<p><code>export HEDWIG_LOG_CONF=/tmp/log4j.properties</code><br />
+<code>hedwig-server/bin/hedwig server</code></p>
</div>
</td>
<td valign="top">
@@ -120,6 +121,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -134,6 +136,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/docs/trunk/index.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/docs/trunk/index.html (original)
+++ websites/production/zookeeper/content/bookkeeper/docs/trunk/index.html Wed Dec 7 14:23:53 2011
@@ -60,6 +60,9 @@
<li><a href="./bookkeeperStarted.html">Getting started</a></li>
<li><a href="./bookkeeperProgrammer.html">Programmer's Guide</a></li>
<li><a href="./bookkeeperConfig.html">Admin Guide</a></li>
+<li><a href="./bookkeeperConfigParams.html">Bookie Server Configuration Parameters</a></li>
+<li><a href="./bookkeeperInternals.html">BookKeeper Internals</a></li>
+<li><a href="./bookieRecovery.html">Bookie Recovery</a></li>
<li><a href="./bookkeeperStream.html">Using BookKeeper stream library</a></li>
</ul>
@@ -88,6 +91,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -102,6 +106,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/index.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/index.html (original)
+++ websites/production/zookeeper/content/bookkeeper/index.html Wed Dec 7 14:23:53 2011
@@ -90,6 +90,7 @@ Learn more about Hedwig on the <a href="
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -104,6 +105,7 @@ Learn more about Hedwig on the <a href="
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/irc.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/irc.html (original)
+++ websites/production/zookeeper/content/bookkeeper/irc.html Wed Dec 7 14:23:53 2011
@@ -76,6 +76,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -90,6 +91,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/lists.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/lists.html (original)
+++ websites/production/zookeeper/content/bookkeeper/lists.html Wed Dec 7 14:23:53 2011
@@ -113,6 +113,7 @@ In order to post to the list, it is nece
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -127,6 +128,7 @@ In order to post to the list, it is nece
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/privacy.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/privacy.html (original)
+++ websites/production/zookeeper/content/bookkeeper/privacy.html Wed Dec 7 14:23:53 2011
@@ -88,6 +88,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -102,6 +103,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/releases.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/releases.html (original)
+++ websites/production/zookeeper/content/bookkeeper/releases.html Wed Dec 7 14:23:53 2011
@@ -53,9 +53,30 @@
<tr>
<td style="overflow: hidden;" valign="top" width="100%">
<div class="wiki-content">
- <h1>Apache ZooKeeper Releases</h1>
+ <h1>Apache BookKeeper Releases</h1>
-<p>No releases yet.</p>
+<ul>
+<li><a href="#download">Download</a></li>
+<li><a href="#releasenotes">Release Notes</a></li>
+<li><a href="#news">News</a></li>
+</ul>
+
+<h2 id="download">Download</h2>
+
+<p>Releases may be downloaded from Apache mirrors: <a href="http://www.apache.org/dyn/closer.cgi/zookeeper/bookkeeper">Download</a></p>
+
+<!--On the mirror, all recent releases are available, but are not guaranteed to be stable. For stable releases, look in the stable directory.//-->
+
+<h2 id="releasenotes">Release Notes</h2>
+
+<p>Release notes for Apache BookKeeper releases are available in Jira: <a href="https://issues.apache.org/jira/browse/BOOKKEEPER?report=com.atlassian.jira.plugin.system.project%3Achangelog-panel%3Aversions-panel">Browse release notes</a></p>
+
+<h2 id="news">News</h2>
+
+<h3>26 Nov, 2011: release 4.0.0 available</h3>
+
+<p>This is the first release of BookKeeper as a subproject of Zookeeper.<br />
+See <a href="./docs/r4.0.0/releaseNotes.html">BookKeeper 4.0.0 Release Notes</a> for details.</p>
</div>
</td>
<td valign="top">
@@ -74,6 +95,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -88,6 +110,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>
Modified: websites/production/zookeeper/content/bookkeeper/svn.html
==============================================================================
--- websites/production/zookeeper/content/bookkeeper/svn.html (original)
+++ websites/production/zookeeper/content/bookkeeper/svn.html Wed Dec 7 14:23:53 2011
@@ -90,6 +90,7 @@
<ul>
<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="/bookkeeper/releases.html">Releases</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
<li><a href="/bookkeeper/credits.html">Credits</a></li>
<li><a href="/bylaws.html">Bylaws</a></li>
@@ -104,6 +105,7 @@
<h3>Documentation</h3>
<ul>
+<li><a href="/bookkeeper/docs/r4.0.0">Release 4.0.0</a></li>
<li><a href="/bookkeeper/docs/trunk">Trunk</a></li>
</ul>