You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by bu...@apache.org on 2011/06/28 04:51:01 UTC
svn commit: r791701 [1/4] - in /websites/staging/zookeeper/trunk/content:
bookkeeper/ bookkeeper/docs/ bookkeeper/docs/trunk/ doc/trunk/
Author: buildbot
Date: Tue Jun 28 02:51:00 2011
New Revision: 791701
Log:
Staging update by buildbot
Added:
websites/staging/zookeeper/trunk/content/bookkeeper/docs/
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperConfig.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperOverview.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStarted.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStream.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/doc.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigBuild.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigDesign.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigUser.html
websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/index.html
Modified:
websites/staging/zookeeper/trunk/content/bookkeeper/index.html
websites/staging/zookeeper/trunk/content/bookkeeper/irc.html
websites/staging/zookeeper/trunk/content/bookkeeper/lists.html
websites/staging/zookeeper/trunk/content/bookkeeper/privacy.html
websites/staging/zookeeper/trunk/content/bookkeeper/releases.html
websites/staging/zookeeper/trunk/content/bookkeeper/svn.html
websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.html
websites/staging/zookeeper/trunk/content/doc/trunk/bookkeeperStarted.pdf
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.html
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperAdmin.pdf
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.html
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperInternals.pdf
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperStarted.html
websites/staging/zookeeper/trunk/content/doc/trunk/zookeeperStarted.pdf
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperConfig.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperConfig.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperConfig.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,168 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - BookKeeper Administrator's Guide</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Abstract</h1>
+
+<p>This document contains information about deploying, administering and mantaining BookKeeper. It also discusses best practices and common problems. </p>
+
+<p>As BookKeeper is still a prototype, this article is likely to change significantly over time. </p>
+
+<h1>System requirements</h1>
+
+<p>A typical BookKeeper installation comprises a set of bookies and a set of ZooKeeper replicas. The exact number of bookies depends on the quorum mode, desired throughput, and number of clients using this installation simultaneously. The minimum number of bookies is three for self-verifying (stores a message authentication code along with each entry) and four for generic (does not store a message authentication codewith each entry), and there is no upper limit on the number of bookies. Increasing the number of bookies, in fact, enables higher throughput. </p>
+
+<p>For performance, we require each server to have at least two disks. It is possible to run a bookie with a single disk, but performance will be significantly lower in this case. Of course, it works with one disk, but performance is significantly lower. </p>
+
+<p>For ZooKeeper, there is no constraint with respect to the number of replicas. Having a single machine running ZooKeeper in standalone mode is sufficient for BookKeeper. For resilience purposes, it might be a good idea to run ZooKeeper in quorum mode with multiple servers. Please refer to the ZooKeeper documentation for detail on how to configure ZooKeeper with multiple replicas </p>
+
+<h1>Running bookies</h1>
+
+<p>To run a bookie, we execute the following command: </p>
+
+<p> <code>java -cp .:./zookeeper-<version>-bookkeeper.jar:./zookeeper-<version>.jar\ :../log4j/apache-log4j-1.2.15/log4j-1.2.15.jar -Dlog4j.configuration=log4j.properties\ org.apache.bookkeeper.proto.BookieServer 3181 127.0.0.1:2181 /path_to_log_device/\ /path_to_ledger_device/ </code> </p>
+
+<p>The parameters are: </p>
+
+<ul>
+<li>Port number that the bookie listens on; </li>
+<li>Comma separated list of ZooKeeper servers with a hostname:port format; </li>
+<li>Path for Log Device (stores bookie write-ahead log); </li>
+<li>Path for Ledger Device (stores ledger entries); </li>
+</ul>
+
+
+<p>Ideally, <code>/path_to_log_device/</code> and <code>/path_to_ledger_device/</code> are each in a different device. </p>
+
+<h1>ZooKeeper Metadata</h1>
+
+<p>For BookKeeper, we require a ZooKeeper installation to store metadata, and to pass the list of ZooKeeper servers as parameter to the constructor of the BookKeeper class ( <code>org.apache.bookkeeper.client,BookKeeper</code> ). To setup ZooKeeper, please check the <a href="http://index.html">ZooKeeper documentation</a>. </p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperOverview.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperOverview.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperOverview.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,247 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - BookKeeper overview</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Abstract</h1>
+
+<p>This guide contains detailed information about using BookKeeper for logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs.</p>
+
+<h1>BookKeeper introduction</h1>
+
+<p>BookKeeper is a replicated service to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers, can crash, corrupt data, discard data, but as long as there are enough bookies behaving correctly the service as a whole behaves correctly. </p>
+
+<p>The initial motivation for BookKeeper comes from the namenode of <span class="caps">HDFS.</span> Namenodes have to log operations in a reliable fashion so that recovery is possible in the case of crashes. We have found the applications for BookKeeper extend far beyond <span class="caps">HDFS, </span>however. Essentially, any application that requires an append storage can replace their implementations with BookKeeper. BookKeeper has the advantage of scaling throughput with the number of servers. </p>
+
+<p>At a high level, a bookkeeper client receives entries from a client application and stores it to sets of bookies, and there are a few advantages in having such a service: </p>
+
+<ul>
+<li>We can use hardware that is optimized for such a service. We currently believe that such a system has to be optimized only for disk I/O; </li>
+<li>We can have a pool of servers implementing such a log system, and shared among a number of servers; </li>
+<li>We can have a higher degree of replication with such a pool, which makes sense if the hardware necessary for it is cheaper compared to the one the application uses. </li>
+</ul>
+
+
+<h1>In slightly more detail...</h1>
+
+<p>BookKeeper implements highly available logs, and it has been designed with write-ahead logging in mind. Besides high availability due to the replicated nature of the service, it provides high throughput due to striping. As we write entries in a subset of bookies of an ensemble and rotate writes across available quorums, we are able to increase throughput with the number of servers for both reads and writes. Scalability is a property that is possible to achieve in this case due to the use of quorums. Other replication techniques, such as state-machine replication, do not enable such a property. </p>
+
+<p>An application first creates a ledger before writing to bookies through a local BookKeeper client instance. Upon creating a ledger, a BookKeeper client writes metadata about the ledger to ZooKeeper. Each ledger currently has a single writer. This writer has to execute a close ledger operation before any other client can read from it. If the writer of a ledger does not close a ledger properly because, for example, it has crashed before having the opportunity of closing the ledger, then the next client that tries to open a ledger executes a procedure to recover it. As closing a ledger consists essentially of writing the last entry written to a ledger to ZooKeeper, the recovery procedure simply finds the last entry written correctly and writes it to ZooKeeper. </p>
+
+<p>Note that currently this recovery procedure is executed automatically upon trying to open a ledger and no explicit action is necessary. Although two clients may try to recover a ledger concurrently, only one will succeed, the first one that is able to create the close znode for the ledger. </p>
+
+<h1>Bookkeeper elements and concepts</h1>
+
+<p>BookKeeper uses four basic elements: </p>
+
+<ul>
+<li> <em>Ledger</em> : A ledger is a sequence of entries, and each entry is a sequence of bytes. Entries are written sequentially to a ledger and at most once. Consequently, ledgers have an append-only semantics; </li>
+<li> <em>BookKeeper client</em> : A client runs along with a BookKeeper application, and it enables applications to execute operations on ledgers, such as creating a ledger and writing to it; </li>
+<li> <em>Bookie</em> : A bookie is a BookKeeper storage server. Bookies store the content of ledgers. For any given ledger L, we call an <em>ensemble</em> the group of bookies storing the content of L. For performance, we store on each bookie of an ensemble only a fragment of a ledger. That is, we stripe when writing entries to a ledger such that each entry is written to sub-group of bookies of the ensemble. </li>
+<li> <em>Metadata storage service</em> : BookKeeper requires a metadata storage service to store information related to ledgers and available bookies. We currently use ZooKeeper for such a task. </li>
+</ul>
+
+
+<h1>Bookkeeper initial design</h1>
+
+<p>A set of bookies implements BookKeeper, and we use a quorum-based protocol to replicate data across the bookies. There are basically two operations to an existing ledger: read and append. Here is the complete <span class="caps">API </span>list (mode detail <a href="http://bookkeeperProgrammer.html">here</a>): </p>
+
+<ul>
+<li>Create ledger: creates a new empty ledger; </li>
+<li>Open ledger: opens an existing ledger for reading; </li>
+<li>Add entry: adds a record to a ledger either synchronously or asynchronously; </li>
+<li>Read entries: reads a sequence of entries from a ledger either synchronously or asynchronously </li>
+</ul>
+
+
+<p>There is only a single client that can write to a ledger. Once that ledger is closed or the client fails, no more entries can be added. (We take advantage of this behavior to provide our strong guarantees.) There will not be gaps in the ledger. Fingers get broken, people get roughed up or end up in prison when books are manipulated, so there is no deleting or changing of entries. </p>
+
+<p><img src="images/bk-overview.jpg" alt="" />
+p. A simple use of BooKeeper is to implement a write-ahead transaction log. A server maintains an in-memory data structure (with periodic snapshots for example) and logs changes to that structure before it applies the change. The application server creates a ledger at startup and store the ledger id and password in a well known place (ZooKeeper maybe). When it needs to make a change, the server adds an entry with the change information to a ledger and apply the change when BookKeeper adds the entry successfully. The server can even use asyncAddEntry to queue up many changes for high change throughput. BooKeeper meticulously logs the changes in order and call the completion functions in order. </p>
+
+<p>When the application server dies, a backup server will come online, get the last snapshot and then it will open the ledger of the old server and read all the entries from the time the snapshot was taken. (Since it doesn't know the last entry number it will use <span class="caps">MAX</span>_INTEGER). Once all the entries have been processed, it will close the ledger and start a new one for its use. </p>
+
+<p>A client library takes care of communicating with bookies and managing entry numbers. An entry has the following fields: </p>
+
+<table><tr><td>Field</td><td>Type</td><td>Description</td></tr><tr><td>Ledger number</td><td>long</td><td>The id of the ledger of this entry</td></tr><tr><td>Entry number</td><td>long</td><td>The id of this entry</td></tr><tr><td>last confirmed ( <em>LC</em> )</td><td>long</td><td>id of the last recorded entry</td></tr><tr><td>data</td><td>byte[]</td><td>the entry data (supplied by application)</td></tr><tr><td>authentication code</td><td>byte[]</td><td>Message authentication code that includes all other fields of the entry</td></tr></table>
+
+
+<p>The client library generates a ledger entry. None of the fields are modified by the bookies and only the first three fields are interpreted by the bookies. </p>
+
+<p>To add to a ledger, the client generates the entry above using the ledger number. The entry number will be one more than the last entry generated. The <em>LC</em> field contains the last entry that has been successfully recorded by BookKeeper. If the client writes entries one at a time, <em>LC</em> is the last entry id. But, if the client is using asyncAddEntry, there may be many entries in flight. An entry is considered recorded when both of the following conditions are met: </p>
+
+<ul>
+<li>the entry has been accepted by a quorum of bookies </li>
+<li>all entries with a lower entry id have been accepted by a quorum of bookies </li>
+</ul>
+
+
+<p> <em>LC</em> seems mysterious right now, but it is too early to explain how we use it; just smile and move on. </p>
+
+<p>Once all the other fields have been field in, the client generates an authentication code with all of the previous fields. The entry is then sent to a quorum of bookies to be recorded. Any failures will result in the entry being sent to a new quorum of bookies. </p>
+
+<p>To read, the client library initially contacts a bookie and starts requesting entries. If an entry is missing or invalid (a bad <span class="caps">MAC </span>for example), the client will make a request to a different bookie. By using quorum writes, as long as enough bookies are up we are guaranteed to eventually be able to read an entry. </p>
+
+<h1>Bookkeeper metadata management</h1>
+
+<p>There are some meta data that needs to be made available to BookKeeper clients: </p>
+
+<ul>
+<li>The available bookies; </li>
+<li>The list of ledgers; </li>
+<li>The list of bookies that have been used for a given ledger; </li>
+<li>The last entry of a ledger; </li>
+</ul>
+
+
+<p>We maintain this information in ZooKeeper. Bookies use ephemeral nodes to indicate their availability. Clients use znodes to track ledger creation and deletion and also to know the end of the ledger and the bookies that were used to store the ledger. Bookies also watch the ledger list so that they can cleanup ledgers that get deleted. </p>
+
+<h1>Closing out ledgers</h1>
+
+<p>The process of closing out the ledger and finding the last ledger is difficult due to the durability guarantees of BookKeeper: </p>
+
+<ul>
+<li>If an entry has been successfully recorded, it must be readable. </li>
+<li>If an entry is read once, it must always be available to be read. </li>
+</ul>
+
+
+<p>If the ledger was closed gracefully, ZooKeeper will have the last entry and everything will work well. But, if the BookKeeper client that was writing the ledger dies, there is some recovery that needs to take place. </p>
+
+<p>The problematic entries are the ones at the end of the ledger. There can be entries in flight when a BookKeeper client dies. If the entry only gets to one bookie, the entry should not be readable since the entry will disappear if that bookie fails. If the entry is only on one bookie, that doesn't mean that the entry has not been recorded successfully; the other bookies that recorded the entry might have failed. </p>
+
+<p>The trick to making everything work is to have a correct idea of a last entry. We do it in roughly three steps: </p>
+
+<ol>
+<li>Find the entry with the highest last recorded entry, <em>LC</em> ; </li>
+<li>Find the highest consecutively recorded entry, <em>LR</em> ; </li>
+<li>Make sure that all entries between <em>LC</em> and <em>LR</em> are on a quorum of bookies; </li>
+</ol>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperProgrammer.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,367 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - BookKeeper Getting Started Guide</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Abstract</h1>
+
+<p>This guide contains detailed information about using BookKeeper for logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs.</p>
+
+<h1> Instantiating BookKeeper.</h1>
+
+<p>The first step to use BookKeeper is to instantiate a BookKeeper object: </p>
+
+<p> <code>org.apache.bookkeeper.BookKeeper </code> </p>
+
+<p>There are three BookKeeper constructors: </p>
+
+<p> <code>public BookKeeper(String servers) throws KeeperException, IOException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>servers</code> is a comma-separated list of ZooKeeper servers. </li>
+</ul>
+
+
+<p> <code>public BookKeeper(ZooKeeper zk) throws InterruptedException, KeeperException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>zk</code> is a ZooKeeper object. This constructor is useful when the application also using ZooKeeper and wants to have a single instance of ZooKeeper. </li>
+</ul>
+
+
+<p> <code>public BookKeeper(ZooKeeper zk, ClientSocketChannelFactory channelFactory) throws InterruptedException, KeeperException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>zk</code> is a ZooKeeper object. This constructor is useful when the application also using ZooKeeper and wants to have a single instance of ZooKeeper. </li>
+<li> <code>channelFactory</code> is a netty channel object ( <code>org.jboss.netty.channel.socket</code> ). </li>
+</ul>
+
+
+<h1> Creating a ledger. </h1>
+
+<p>Before writing entries to BookKeeper, it is necessary to create a ledger. With the current BookKeeper <span class="caps">API, </span>it is possible to create a ledger both synchronously or asynchronously. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
+
+<p> <em>Synchronous call:</em> </p>
+
+<p> <code>public LedgerHandle createLedger(int ensSize, int qSize, DigestType type, byte passwd[]) throws KeeperException, InterruptedException, IOException, BKException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>ensSize</code> is the number of bookies (ensemble size); </li>
+<li> <code>qSize</code> is the write quorum size; </li>
+<li> <code>type</code> is the type of digest used with entries: either <span class="caps">MAC </span>or <span class="caps">CRC32. </span></li>
+<li> <code>passwd</code> is a password that authorizes the client to write to the ledger being created. </li>
+</ul>
+
+
+<p>All further operations on a ledger are invoked through the <code>LedgerHandle</code> object returned. </p>
+
+<p>As a convenience, we provide a <code>createLedger</code> with default parameters (3,2,VERIFIABLE), and the only two input parameters it requires are a digest type and a password. </p>
+
+<p> <em>Asynchronous call:</em> </p>
+
+<p> <code>public void asyncCreateLedger(int ensSize, int qSize, DigestType type, byte passwd[], CreateCallback cb, Object ctx ) </code> </p>
+
+<p>The parameters are the same of the synchronous version, with the exception of <code>cb</code> and <code>ctx</code> . <code>CreateCallback</code> is an interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>createComplete</code> that has the following signature: </p>
+
+<p> <code>void createComplete(int rc, LedgerHandle lh, Object ctx); </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKException</code> for a list); </li>
+<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
+<li> <code>ctx</code> is a control object for accountability purposes. It can be essentially any object the application is happy with. </li>
+</ul>
+
+
+<p>The <code>ctx</code> object passed as a parameter to the call to create a ledger is the one same returned in the callback. </p>
+
+<h1> Adding entries to a ledger. </h1>
+
+<p>Once we have a ledger handle <code>lh</code> obtained through a call to create a ledger, we can start writing entries. As with creating ledgers, we can write both synchronously and asynchronously. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
+
+<p> <em>Synchronous call:</em> </p>
+
+<p> <code>public long addEntry(byte[] data) throws InterruptedException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>data</code> is a byte array; </li>
+</ul>
+
+
+<p>A call to <code>addEntry</code> returns the status of the operation (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </p>
+
+<p> <em>Asynchronous call:</em> </p>
+
+<p> <code>public void asyncAddEntry(byte[] data, AddCallback cb, Object ctx) </code> </p>
+
+<p>It also takes a byte array as the sequence of bytes to be stored as an entry. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>AddCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>addComplete</code> that has the following signature: </p>
+
+<p> <code>void addComplete(int rc, LedgerHandle lh, long entryId, Object ctx); </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
+<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
+<li> <code>entryId</code> is the identifier of entry associated with this request; </li>
+<li> <code>ctx</code> is control object used for accountability purposes. It can be any object the application is happy with. </li>
+</ul>
+
+
+<h1> Closing a ledger. </h1>
+
+<p>Once a client is done writing, it closes the ledger. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
+
+<p> <em>Synchronous close:</em> </p>
+
+<p> <code>public void close() throws InterruptedException </code> </p>
+
+<p>It takes no input parameters. </p>
+
+<p> <em>Asynchronous close:</em> </p>
+
+<p> <code>public void asyncClose(CloseCallback cb, Object ctx) throws InterruptedException </code> </p>
+
+<p>It takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>CloseCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>closeComplete</code> that has the following signature: </p>
+
+<p> <code>void closeComplete(int rc, LedgerHandle lh, Object ctx) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
+<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
+<li> <code>ctx</code> is control object used for accountability purposes. </li>
+</ul>
+
+
+<h1> Opening a ledger. </h1>
+
+<p>To read from a ledger, a client must open it first. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
+
+<p> <em>Synchronous open:</em> </p>
+
+<p> <code>public LedgerHandle openLedger(long lId, DigestType type, byte passwd[]) throws InterruptedException, BKException </code> </p>
+
+<ul>
+<li> <code>ledgerId</code> is the ledger identifier; </li>
+<li> <code>type</code> is the type of digest used with entries: either <span class="caps">MAC </span>or <span class="caps">CRC32. </span></li>
+<li> <code>passwd</code> is a password to access the ledger (used only in the case of <code>VERIFIABLE</code> ledgers); </li>
+</ul>
+
+
+<p> <em>Asynchronous open:</em> </p>
+
+<p> <code>public void asyncOpenLedger(long lId, DigestType type, byte passwd[], OpenCallback cb, Object ctx) </code> </p>
+
+<p>It also takes a a ledger identifier and a password. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>OpenCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>openComplete</code> that has the following signature: </p>
+
+<p> <code>public void openComplete(int rc, LedgerHandle lh, Object ctx) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
+<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
+<li> <code>ctx</code> is control object used for accountability purposes. </li>
+</ul>
+
+
+<h1> Reading from ledger </h1>
+
+<p>Read calls may request one or more consecutive entries. The following methods belong to <code>org.apache.bookkeeper.client.LedgerHandle</code> . </p>
+
+<p> <em>Synchronous read:</em> </p>
+
+<p> <code>public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry) throws InterruptedException, BKException </code> </p>
+
+<ul>
+<li> <code>firstEntry</code> is the identifier of the first entry in the sequence of entries to read; </li>
+<li> <code>lastEntry</code> is the identifier of the last entry in the sequence of entries to read. </li>
+</ul>
+
+
+<p> <em>Asynchronous read:</em> </p>
+
+<p> <code>public void asyncReadEntries(long firstEntry, long lastEntry, ReadCallback cb, Object ctx) throws BKException, InterruptedException </code> </p>
+
+<p>It also takes a first and a last entry identifiers. Additionaly, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>ReadCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>readComplete</code> that has the following signature: </p>
+
+<p> <code>void readComplete(int rc, LedgerHandle lh, Enumeration<LedgerEntry> seq, Object ctx) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
+<li> <code>lh</code> is a <code>LedgerHandle</code> object to manipulate a ledger; </li>
+<li> <code>seq</code> is a <code>Enumeration<LedgerEntry></code> object to containing the list of entries requested; </li>
+<li> <code>ctx</code> is control object used for accountability purposes. </li>
+</ul>
+
+
+<h1> Deleting a ledger </h1>
+
+<p>Once a client is done with a ledger and is sure that nobody will ever need to read from it again, they can delete the ledger. The following methods belong to <code>org.apache.bookkeeper.client.BookKeeper</code> . </p>
+
+<p> <em>Synchronous delete:</em> </p>
+
+<p> <code>public void deleteLedger(long lId) throws InterruptedException, BKException </code> </p>
+
+<ul>
+<li> <code>lId</code> is the ledger identifier; </li>
+</ul>
+
+
+<p> <em>Asynchronous delete:</em> </p>
+
+<p> <code>public void asyncDeleteLedger(long lId, DeleteCallback cb, Object ctx) </code> </p>
+
+<p>It takes a ledger identifier. Additionally, it takes a callback object <code>cb</code> and a control object <code>ctx</code> . The callback object must implement the <code>DeleteCallback</code> interface in <code>org.apache.bookkeeper.client.AsyncCallback</code> , and a class implementing it has to implement a method called <code>deleteComplete</code> that has the following signature: </p>
+
+<p> <code>void deleteComplete(int rc, Object ctx) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>rc</code> is a return code (please refer to <code>org.apache.bookeeper.client.BKDefs</code> for a list); </li>
+<li> <code>ctx</code> is control object used for accountability purposes. </li>
+</ul>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStarted.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStarted.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStarted.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,214 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - BookKeeper Getting Started Guide</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Abstract</h1>
+
+<p>This guide contains detailed information about using BookKeeper for logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs.</p>
+
+<h1>Getting Started: Setting up BookKeeper to write logs.</h1>
+
+<p>This document contains information to get you started quickly with BookKeeper. It is aimed primarily at developers willing to try it out, and contains simple installation instructions for a simple BookKeeper installation and a simple programming example. For further programming detail, please refer to <a href="http://bookkeeperProgrammer.html">BookKeeper Programmer's Guide</a>. </p>
+
+<h1>Pre-requisites</h1>
+
+<p>See <a href="http://bookkeeperConfig.html#bk_sysReqin">System Requirements</a> the Admin guide.</p>
+
+<h1>Download</h1>
+
+<p>BookKeeper is distributed along with ZooKeeper. To get a ZooKeeper distribution, download a recent <a href="http://hadoop.apache.org/zookeeper/releases.htmlrelease">stable</a> from one of the Apache Download Mirrors.</p>
+
+<h1>LocalBookKeeper</h1>
+
+<p>Under org.apache.bookkeeper.util, you'll find a java program called LocalBookKeeper.java that sets you up to run BookKeeper on a single machine. This is far from ideal from a performance perspective, but the program is useful for both test and educational purposes. </p>
+
+<h1>Setting up bookies</h1>
+
+<p>If you're bold and you want more than just running things locally, then you'll need to run bookies in different servers. You'll need at least three bookies to start with. </p>
+
+<p>For each bookie, we need to execute a command like the following: </p>
+
+<p> <code>java -cp .:./zookeeper-<version>-bookkeeper.jar:./zookeeper-<version>.jar\ :lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar -Dlog4j.configuration=log4j.properties\ org.apache.bookkeeper.proto.BookieServer 3181 127.0.0.1:2181 /path_to_log_device/\ /path_to_ledger_device/ </code> </p>
+
+<p>"/path_to_log_device/" and "/path_to_ledger_device/" are different paths. Also, port 3181 is the port that a bookie listens on for connection requests from clients. 127.0.0.1:2181 is the hostname:port for the ZooKeeper server. In this example, the standalone ZooKeeper server is running locally on port 2181. If we had multiple ZooKeeper servers, this parameter would be a comma separated list of all the hostname:port values corresponding to them. </p>
+
+<h1>Setting up ZooKeeper</h1>
+
+<p>ZooKeeper stores metadata on behalf of BookKeeper clients and bookies. To get a minimal ZooKeeper installation to work with BookKeeper, we can set up one server running in standalone mode. Once we have the server running, we need to create a few znodes: </p>
+
+<ol>
+<li> <code>/ledgers </code> </li>
+<li> <code>/ledgers/available </code> </li>
+<li>For each bookie, we add one znode such that the name of the znode is the concatenation of the machine name and the port number that the bookie is listening on. For example, if a bookie is running on bookie.foo.com an is listening on port 3181, we add a znode <code>/ledgers/available/bookie.foo.com:3181</code> . </li>
+</ol>
+
+
+<h1>Example</h1>
+
+<p>In the following excerpt of code, we: </p>
+
+<ol>
+<li>Create a ledger; </li>
+<li>Write to the ledger; </li>
+<li>Close the ledger; </li>
+<li>Open the same ledger for reading; </li>
+<li>Read from the ledger; </li>
+<li>Close the ledger again; </li>
+</ol>
+
+<pre><code>
+LedgerHandle lh = bkc.createLedger(ledgerPassword);
+ledgerId = lh.getId();
+ByteBuffer entry = ByteBuffer.allocate(4);
+
+for(int i = 0; i < 10; i++){
+ entry.putInt(i);
+ entry.position(0);
+ entries.add(entry.array());
+ lh.addEntry(entry.array());
+}
+lh.close();
+lh = bkc.openLedger(ledgerId, ledgerPassword);
+
+Enumeration<LedgerEntry> ls = lh.readEntries(0, 9);
+int i = 0;
+while(ls.hasMoreElements()){
+ ByteBuffer origbb = ByteBuffer.wrap(
+ entries.get(i++));
+ Integer origEntry = origbb.getInt();
+ ByteBuffer result = ByteBuffer.wrap(
+ ls.nextElement().getEntry());
+
+ Integer retrEntry = result.getInt();
+}
+lh.close();
+ </code></pre>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStream.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStream.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/bookkeeperStream.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,262 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - Streaming with BookKeeper</title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Abstract</h1>
+
+<p>This guide contains detailed information about using how to stream bytes on top of BookKeeper. It essentially motivates and discusses the basic stream operations currently supported.</p>
+
+<h1>Summary</h1>
+
+<p>When using the BookKeeper <span class="caps">API, </span>an application has to split the data to write into entries, each entry being a byte array. This is natural for many applications. For example, when using BookKeeper for write-ahead logging, an application typically wants to write the modifications corresponding to a command or a transaction. Some other applications, however, might not have a natural boundary for entries, and may prefer to write and read streams of bytes. This is exactly the purpose of the stream <span class="caps">API </span>we have implemented on top of BookKeeper. </p>
+
+<p>The stream <span class="caps">API </span>is implemented in the package <code>Streaming</code> , and it contains two main classes: <code>LedgerOutputStream</code> and <code>LedgerInputStream</code> . The class names are indicative of what they do. </p>
+
+<h1>Writing a stream of bytes</h1>
+
+<p>Class <code>LedgerOutputStream</code> implements two constructors and five public methods: </p>
+
+<p> <code>public LedgerOutputStream(LedgerHandle lh) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+</ul>
+
+
+<p> <code>public LedgerOutputStream(LedgerHandle lh, int size) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+<li> <code>size</code> is the size of the byte buffer to store written bytes before flushing. </li>
+</ul>
+
+
+<p> <em>Closing a stream.</em> This call closes the stream by flushing the write buffer. </p>
+
+<p> <code>public void close() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Flushing a stream.</em> This call essentially flushes the write buffer. </p>
+
+<p> <code>public synchronized void flush() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Writing bytes.</em> There are three calls for writing bytes to a stream. </p>
+
+<p> <code>public synchronized void write(byte[] b) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is an array of bytes to write. </li>
+</ul>
+
+
+<p> <code>public synchronized void write(byte[] b, int off, int len) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is an array of bytes to write. </li>
+<li> <code>off</code> is a buffer offset. </li>
+<li> <code>len</code> is the length to write. </li>
+</ul>
+
+
+<p> <code>public synchronized void write(int b) </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> contains a byte to write. The method writes the least significant byte of the integer four bytes. </li>
+</ul>
+
+
+<h1>Reading a stream of bytes</h1>
+
+<p>Class <code>LedgerOutputStream</code> implements two constructors and four public methods: </p>
+
+<p> <code>public LedgerInputStream(LedgerHandle lh) throws BKException, InterruptedException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+</ul>
+
+
+<p> <code>public LedgerInputStream(LedgerHandle lh, int size) throws BKException, InterruptedException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>lh</code> is a ledger handle for a previously created and open ledger. </li>
+<li> <code>size</code> is the size of the byte buffer to store bytes that the application will eventually read. </li>
+</ul>
+
+
+<p> <em>Closing.</em> There is one call to close an input stream, but the call is currently empty and the application is responsible for closing the ledger handle. </p>
+
+<p> <code>public void close() </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <em>Reading.</em> There are three calls to read from the stream. </p>
+
+<p> <code>public synchronized int read() throws IOException </code> </p>
+
+<p>which has no parameters. </p>
+
+<p> <code>public synchronized int read(byte[] b) throws IOException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is a byte array to write to. </li>
+</ul>
+
+
+<p> <code>public synchronized int read(byte[] b, int off, int len) throws IOException </code> </p>
+
+<p>where: </p>
+
+<ul>
+<li> <code>b</code> is a byte array to write to. </li>
+<li> <code>off</code> is an offset for byte array <code>b</code> . </li>
+<li> <code>len</code> is the length in bytes to write to <code>b</code> . </li>
+</ul>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/doc.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/doc.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/doc.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,144 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - </title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <p>In the documentation directory, you'll find:</p>
+
+<ul>
+<li><code>build.txt</code>: Building Hedwig, or how to set up Hedwig</li>
+<li><code>user.txt</code>: User's Guide, or how to program against the Hedwig <span class="caps">API </span>and how to run it</li>
+<li><code>dev.txt</code>: Developer's Guide, or Hedwig internals and hacking details</li>
+</ul>
+
+<p>These documents are all written in the <a href="http://johnmacfarlane.net/pandoc/">Pandoc</a> dialect of <a href="http://daringfireball.net/projects/markdown/">Markdown</a>. This makes them readable as plain text files, but also capable of generating <span class="caps">HTML </span>or LaTeX documentation.</p>
+
+<p>Documents are wrapped at 80 chars and use 2-space indentation.</p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>
Added: websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigBuild.html
==============================================================================
--- websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigBuild.html (added)
+++ websites/staging/zookeeper/trunk/content/bookkeeper/docs/trunk/hedwigBuild.html Tue Jun 28 02:51:00 2011
@@ -0,0 +1,230 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
+<html>
+<head>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+ <link href="css/site.css" rel="stylesheet" type="text/css">
+ <link href="css/type-settings.css" rel="stylesheet" type="text/css">
+
+ <title>Apache BookKeeper - </title>
+
+ <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+</head>
+<body>
+<div class="white_box">
+<div class="header">
+ <div class="header_l">
+ <div class="header_r">
+ </div>
+ </div>
+</div>
+<div class="content">
+ <div class="content_l">
+ <div class="content_r">
+ <div>
+
+<!-- Banner -->
+
+
+ <div id="header_background">
+ <div id="bookkeeper_logo">
+ <a href="/"><img src="images/bookkeeper_small.gif"/></a> <b><font size=+4 color=white>Apache BookKeeper™</font></b> <a href="http://apache.org"><img src="images/feather_small.gif"/></a>
+ </div>
+ </div>
+
+ <table border="0">
+ <tbody>
+ <tr>
+ <td style="overflow: hidden;" valign="top" width="100%">
+ <div class="wiki-content">
+ <h1>Pre-requisites</h1>
+
+<p>For the core itself:</p>
+
+<ul>
+<li><span class="caps">JDK</span> 6: <a href="http://java.sun.com/">http://java.sun.com/</a>. Ensure <code>$JAVA_HOME</code> is correctly set.</li>
+<li>Maven 2: <a href="http://maven.apache.org/">http://maven.apache.org/</a>.</li>
+<li>Protocol Buffers 2.3.0: <a href="http://protobuf.googlecode.com/">http://protobuf.googlecode.com/</a>.</li>
+<li>Zookeeper 3.4.0: <a href="http://hadoop.apache.org/zookeeper/">http://hadoop.apache.org/zookeeper/</a>. See below.</li>
+<li>Bookkeeper 3.4.0: <a href="http://hadoop.apache.org/zookeeper/">http://hadoop.apache.org/zookeeper/</a>. See below.</li>
+</ul>
+
+<p>Hedwig has been tested on Windows <span class="caps">XP,</span> Linux 2.6, and OS X.</p>
+
+<p>For the deployment and distributed support scripts in <code>hw.bash</code>:</p>
+
+<ul>
+<li>Ant: <a href="http://ant.apache.org/">http://ant.apache.org/</a>, if you want to build Zookeeper.</li>
+<li>Bash: <a href="http://www.gnu.org/software/bash/">http://www.gnu.org/software/bash/</a>.</li>
+<li>Coreutils: <a href="http://www.gnu.org/software/coreutils/">http://www.gnu.org/software/coreutils/</a>.</li>
+<li>Expect: <a href="http://expect.nist.gov/">http://expect.nist.gov/</a>, if you want <code>unbuffer</code>.</li>
+<li>Findutils: <a href="http://www.gnu.org/software/findutils/">http://www.gnu.org/software/findutils/</a>.</li>
+<li>OpenSSH: <a href="http://www.openssh.com/">http://www.openssh.com/</a>.</li>
+<li>Python 2.6: <a href="http://python.org/">http://python.org/</a>.</li>
+</ul>
+
+<h2>Protocol Buffers</h2>
+
+<p>Hedwig requires the use of the Java runtime libraries of Protocol Buffers 2.3.0. These libraries need to be installed into your local maven repository. (Maven allows multiple versions to be installed.) To install protocol buffels to your local repository, you have to download the tarball and follow the <span class="caps">README.</span>txt instructions. Note that you must first install the C++ package which contains the compiler (protoc) before you can build the java libraries. That will install the library jar's in the local maven repository where Hedwig is currently configured to point to.</p>
+
+<h2>Zookeeper and Bookkeeper</h2>
+
+<p>Hedwig currently requires the version of Bookkeeper maintained in Apache's current trunk <span class="caps">SVN </span>respository (version 3.4.0). This is not a released version yet but certain features needed for BookKeeper are only available there.</p>
+
+<p>Hedwig also depends on ZK testing code for its own testing code.</p>
+
+<p>Since Hedwig is a Maven project, all these dependencies must be made available as Maven artifacts. However, neither ZK nor BK are currently Mavenized. Hedwig provides some bash scripts to ease the installation of <span class="caps">ZK,</span> ZK tests, and <span class="caps">BK, </span>all as Maven artifacts.</p>
+
+<p>Currently, we have included the necessary ZooKeeper and BookKeeper jars in the Hedwig source itself in the $HEDWIG_DIR/server/lib directory. There is no need to retrieve them directly from the Apache download site as they are non-released trunk versions.</p>
+
+<h1>Not relevant right now since we already have the ZK jars already in the Hedwig source.</h1>
+
+<p>To fetch and build ZK 3.4.0 (and its tests) in the current directory, run:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash get-zk</p>
+
+<h1>Not relevant right now, but when we start using the apache version of <span class="caps">BK, </span>to</h1>
+
+<p>build the local version of BK:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash get-bk</p>
+
+<p>The $HEDWIG_DIR/server/lib directory contains all of the the class and source jars for <span class="caps">ZK,</span> ZK tests, and <span class="caps">BK.</span> To install these, go to that directory and run the following command to install them into your local maven repository:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash install-zk-bk</p>
+
+<h1>Command-Line Instructions</h1>
+
+<p>From the main Hedwig directory, run <code>mvn package</code>. This will produce the executable jars for both the client and server, as well as a server "assembly jar" containing all dependencies as well for easier deployment.</p>
+
+<p>See the User's Guide for instructions on running and usage.</p>
+
+<h1>Eclipse Instructions</h1>
+
+<p>To check out, build, and develop using Eclipse:</p>
+
+<ol>
+<li>Install the Subclipse plugin. Update site: <a href="http://subclipse.tigris.org/update_1.4.x">http://subclipse.tigris.org/update_1.4.x</a>.</li>
+<li>Install the Maven plugin. Update site: <a href="http://m2eclipse.sonatype.org/update">http://m2eclipse.sonatype.org/update</a>. From the list of packages available from this site, select everything under the "Maven Integration" category, and from the optional components select the ones with the word "SCM" in them.</li>
+<li>Go to Preferences > Team > <span class="caps">SVN.</span> For the <span class="caps">SVN </span>interface, choose "Pure Java".</li>
+<li>Choose File > New > Project... > Maven > Checkout Maven Projects from <span class="caps">SCM.</span></li>
+<li>For the <span class="caps">SCM URL </span>type, choose <span class="caps">SVN.</span> For the <span class="caps">URL, </span>enter <span class="caps">SVN URL.</span> Maven will automatically create a top-level Eclipse project for each of the 4 Maven modules (recommended). If you want fewer top-level projects, uncheck the option of having a project for each module (under Advanced).</li>
+<li>Right-click on the <code>protocol</code> project and choose Run As > Maven generate-sources. This will generate the Java and C++ code for Protocol Buffers.</li>
+<li>Refresh the workspace to pick up the generated code and add <code>hedwig/protocol/target/generated-sources/java</code> as a source folder. (6 & 7 should really be doable automatically, but I haven't figured out how.)</li>
+</ol>
+
+<p>You are now ready to run and debug the client and server code. See the User's Guide for instructions on running and usage.</p>
+
+<h1>Utilities</h1>
+
+<h2>Removing Conflicting Files in Jars</h2>
+
+<p>The Maven assembly plugin that produces the fat assembly jar may end up putting into the jar files with the same conflicting paths from multiple dependencies. This makes working with the files from certain tools (like <code>jar</code>) a bit jarring. In our case, these files are not things like class files, but rather <span class="caps">README </span>and <span class="caps">LICENSE </span>files, so we can safely remove conflicts by choosing an arbitrary winner. To do so, run:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash strip-jar</p>
+
+<h2>Adjusting Logging</h2>
+
+<p>The logging level is something that is baked into the jar in the <code>log4j.properties</code> resource. However, it would be wasteful to go through a Maven build cycle to update and adjust this. If you're working from a source tree, it's also annoying to have to edit a source file to adjust the logging.</p>
+
+<p>We have a little script for tweaking the logging level. After running <code>strip-jar</code>, run:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash set-logging <span class="caps">WARN</span></p>
+
+<p>To see what the current logging level is:</p>
+
+<p>$HEDWIG_DIR/scripts/hw.bash get-logging</p>
+ </div>
+ </td>
+ <td valign="top">
+ <div class="navigation">
+ <div class="navigation_top">
+<div class="searchbox">
+<form action="http://search-hadoop.com/zookeeper" method="get">
+<input onFocus="getBlank (this, 'Search with Apache Solr');" size="25" name="q" id="query" type="text" value="Search with Apache Solr">
+ <input value="Search" type="submit">
+</form>
+</div>
+
+ <div class="navigation_bottom">
+<!-- sidenav -->
+<h3>Project</h3>
+
+<ul>
+<li><a href="/bookkeeper/releases.html#news">News</a></li>
+<li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li>
+<li><a href="/bookkeeper/credits.html">Credits</a></li>
+<li><a href="/bylaws.html">Bylaws</a></li>
+<li><a href="http://www.apache.org/licenses/">License</a></li>
+<li><a href="/privacy.html">Privacy Policy</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+<li><a href="http://www.apache.org/security/">Security</a></li>
+</ul>
+
+<h3>Developers</h3>
+
+<ul>
+<li><a href="/bookkeeper/lists.html">Mailing Lists</a></li>
+<li><a href="/bookkeeper/irc.html"><span class="caps">IRC</span> Channel</a></li>
+<li><a href="/bookkeeper/svn.html">Version Control</a></li>
+<li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li>
+</ul>
+<!-- sidenav -->
+ </div>
+ </div>
+ </div>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+
+<!--
+ <div class="bottom_red_bar"></div>
+-->
+ </div>
+ </div>
+ </div>
+</div>
+<div class="black_box">
+<div class="footer">
+ <div class="footer_l">
+ <div class="footer_r">
+ <div>
+
+ </div>
+ </div>
+ </div>
+</div>
+</div>
+</div>
+<!--
+<div class="design_attribution">Page Template Design By Marc Prud'hommeaux based on <a href="http://activemq.apache.org/">ActiveMQ template</a></div>
+-->
+<div class="copyright_footer">
+<p>Copyright © 2010 The Apache Software Foundation, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.<br>Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache ZooKeeper project logo are trademarks of The Apache Software Foundation.</p>
+</div>
+<script src="skeleton_files/urchin.js" type="text/javascript">
+</script>
+<script type="text/javascript">
+_uacct = "UA-1940143-1";
+urchinTracker();
+</script>
+
+</body></html>