You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Luca Telloli (JIRA)" <ji...@apache.org> on 2009/01/20 12:55:59 UTC
[jira] Created: (ZOOKEEPER-276) Bookkeeper contribution
Bookkeeper contribution
-----------------------
Key: ZOOKEEPER-276
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
Project: Zookeeper
Issue Type: New Feature
Reporter: Luca Telloli
BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (ZOOKEEPER-276) Bookkeeper
contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668718#action_12668718 ]
mahadev edited comment on ZOOKEEPER-276 at 1/29/09 6:50 PM:
------------------------------------------------------------------
also ant tar on top level dir fails with the following error -
{noformat}
compile:
[echo] contrib: bookkeeper
[javac] Compiling 26 source files to /Users/mahadev/workspace/zookeeper-commit-trunk/build/contrib/bookkeeper/classes
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/bookie/Bookie.java:302: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieClient.java:317: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:87: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:164: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/util/LocalBookKeeper.java:143: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] 5 errors
{noformat}
was (Author: mahadev):
also ant tar on top level dir gives the following error -
{noformat}
compile:
[echo] contrib: bookkeeper
[javac] Compiling 26 source files to /Users/mahadev/workspace/zookeeper-commit-trunk/build/contrib/bookkeeper/classes
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/bookie/Bookie.java:302: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieClient.java:317: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:87: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:164: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/util/LocalBookKeeper.java:143: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] 5 errors
{noformat}
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-276:
------------------------------------
Attachment: ZOOKEEPER-276.patch
i fixed some problems in the build.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-276:
-----------------------------------
Fix Version/s: 3.1.0
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668193#action_12668193 ]
Mahadev konar commented on ZOOKEEPER-276:
-----------------------------------------
one thing i noticed -
sorry I should have pointed this out earlier -
we should remove the @author tags.
This is just a convention that hadoop has been following (given that a file might get changed often by different people over time)
example -
{noformat}
+ * @author luca telloli */
in src/contrib/bookkeeper/test/org/apache/bookkeeper/test/BookieReadWriteTest.java
{noformat}
also, their are no javadocs in the classes. I think it will be good to have javadocs for some of the important classes in BookKeeper.
other than that I think the contribution looks good.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luca Telloli updated ZOOKEEPER-276:
-----------------------------------
Attachment: (was: ZOOKEEPER-276.patch)
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-276:
--------------------------------------
Assignee: Flavio Paiva Junqueira
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-276:
------------------------------------
Attachment: ZOOKEEPER-276.patch
fixed license on oen of the files...
and fixed bookkeeper tests ... (they had @override tags which were nto required.)
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671584#action_12671584 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-276:
--------------------------------------------------
Dhruba, That's an excellent question. We are still to document it in detail. Regarding your question, writes are reliable: when a bookie acknowledges receipt of an entry, it is on disk.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (ZOOKEEPER-276) Bookkeeper
contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668786#action_12668786 ]
fpj edited comment on ZOOKEEPER-276 at 1/30/09 1:24 AM:
---------------------------------------------------------------------------
On issue 1: we have checked with Patrick Hunt (and Patrick checked with Doug Cutting) that it is ok to use org.apache.bookkeeper. I also prefer it that way;
On issue 2: Unfortunately my compiler doesn't complain about the override tags you mentioned (I'm on macos, jvm 1.6.0), so I just removed the ones you pointed out. Please give it another try.
was (Author: fpj):
On issue 1: we have checked with Patrick Hunt (and Patrick checked with Doug Cutting) that it is ok to use org.apache.bookkeeper). I also prefer it that way;
On issue 2: Unfortunately my compiler doesn't complain about the override tags you mentioned (I'm on macos, jvm 1.6.0), so I just removed the ones you pointed out. Please give it another try.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-276:
---------------------------------------------
Attachment: ZOOKEEPER-276.patch
On issue 1: we have checked with Patrick Hunt (and Patrick checked with Doug Cutting) that it is ok to use org.apache.bookkeeper). I also prefer it that way;
On issue 2: Unfortunately my compiler doesn't complain about the override tags you mentioned (I'm on macos, jvm 1.6.0), so I just removed the ones you pointed out. Please give it another try.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668951#action_12668951 ]
Mahadev konar commented on ZOOKEEPER-276:
-----------------------------------------
bq. On issue 1: we have checked with Patrick Hunt (and Patrick checked with Doug Cutting) that it is ok to use org.apache.bookkeeper. I also prefer it that way;
That is fine with me.
bq. On issue 2: Unfortunately my compiler doesn't complain about the override tags you mentioned (I'm on macos, jvm 1.6.0), so I just removed the ones you pointed out. Please give it another try.
. will give it a try...
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669575#action_12669575 ]
Hudson commented on ZOOKEEPER-276:
----------------------------------
Integrated in ZooKeeper-trunk #217 (See [http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/217/])
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-276:
---------------------------------------------
Attachment: ZOOKEEPER-276.patch
Implemented Mahadev suggestions.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668718#action_12668718 ]
Mahadev konar commented on ZOOKEEPER-276:
-----------------------------------------
also ant tar on top level dir gives the following error -
{noformat}
compile:
[echo] contrib: bookkeeper
[javac] Compiling 26 source files to /Users/mahadev/workspace/zookeeper-commit-trunk/build/contrib/bookkeeper/classes
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/bookie/Bookie.java:302: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieClient.java:317: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:87: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/proto/BookieServer.java:164: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] /Users/mahadev/workspace/zookeeper-commit-trunk/src/contrib/bookkeeper/src/java/org/apache/bookkeeper/util/LocalBookKeeper.java:143: method does not override a method from its superclass
[javac] @Override
[javac] ^
[javac] 5 errors
{noformat}
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668228#action_12668228 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-276:
--------------------------------------------------
Forgot to mention that unit tests pass just fine for me.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-276:
---------------------------------------------
Attachment: ZOOKEEPER-276.patch
Removed some calls to registerAsConcrete that were preventing me from compiling successfully.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-276:
---------------------------------------------
Attachment: ZOOKEEPER-276.patch
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665412#action_12665412 ]
Luca Telloli commented on ZOOKEEPER-276:
----------------------------------------
BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
The main motivation for this system comes from the namenode of the Hadoop Distributed File System (HDFS). Namenodes have to log operations in a reliable fashion so that recovery is possible in the case of failures. Currently, HDFS does write-ahead logging on its local storage. This allows the namenode to restart after a failure, but it does not allow recovery from a complete machine failure. A shared, fault-tolerant write-ahead log is needed so that the namenode can be started on another machine and recover state properly. We have found the applications for BookKeeper extend beyond HDFS. In fact, any application that uses write-ahead logging can take advantage of BookKeeper.
Back to the namenode example, one potential solution is to replicate across a number of replicas, and use an agreement protocol (e.g., 3PC) to guarantee that operations hit enough replicas. This approach doesn't take full advantage of I/O parallelism that can be achieved by striping. We can also take advantage of a single writer and immutable ledger entries to simplify the protocol and obtain better performance and fault-tolerance guarantees.
The approach we propose is different. We propose a highly-available system that receives entries from a client and stores it. Such a system can be used by several client systems, and not only HDFS. There are several advantages in building such a service:
* We can use hardware that is optimized for such a service. We currently believe that such a system has to be optimized only for disk I/O;
* We can have a pool of servers implementing such a log system, and shared among a number of servers. BookKeeper is hence a good candidate for a cloud service;
* We can have a higher degree of replication with such a pool, which makes sense if the hardware necessary for it is cheaper compared to the one the application uses.
In this patch, we include the source code of BookKeeper to be included in the "contrib" folder of the ZooKeeper distribution. We note that this is still experimental code, but the community is free to evaluate, discuss, and hopefully contribute.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668991#action_12668991 ]
Mahadev konar commented on ZOOKEEPER-276:
-----------------------------------------
+1 on the patch ..
the patch has tests and readme.
some more javadocs would be great but we can fix that later.
ill commit this for now.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luca Telloli updated ZOOKEEPER-276:
-----------------------------------
Status: Patch Available (was: Open)
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-276:
------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks Luca and Flavio.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luca Telloli updated ZOOKEEPER-276:
-----------------------------------
Attachment: ZOOKEEPER-276.patch
Initial contribution of BookKeeper. Please note this is still experimental code.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671566#action_12671566 ]
dhruba borthakur commented on ZOOKEEPER-276:
--------------------------------------------
I have a simple question about bookkeeper ( I could not find design doc, if there is one, please point me to it).
Suppose I have a service with 3 bookies and 1 ledger. I write 1 ledger entry. When this appliction-write to the ledger entry is complete, this transaction is successfully done on at least 2 of the bookies. Does the completion of the application-write also guarantee that this transaction is synchonously persisted on the disk on these two bookies, or does this transaction reside in memory of these two bookies and is lazily flushed? Thanks.
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Luca Telloli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luca Telloli updated ZOOKEEPER-276:
-----------------------------------
Attachment: ZOOKEEPER-276.patch
Last updated version
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-276) Bookkeeper contribution
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668716#action_12668716 ]
Mahadev konar commented on ZOOKEEPER-276:
-----------------------------------------
bookkeeper is contributed as
org.apache.bookkeeper package --
shouldnt it be org.apache.zookeeper.bookkeeper.* ?
> Bookkeeper contribution
> -----------------------
>
> Key: ZOOKEEPER-276
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-276
> Project: Zookeeper
> Issue Type: New Feature
> Reporter: Luca Telloli
> Assignee: Flavio Paiva Junqueira
> Fix For: 3.1.0
>
> Attachments: ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch, ZOOKEEPER-276.patch
>
>
> BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.