You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Patrick Hunt <ph...@apache.org> on 2009/02/13 22:50:47 UTC
[ANNOUNCE] Apache ZooKeeper 3.1.0
The Apache ZooKeeper team is proud to announce Apache ZooKeeper version
3.1.0.
ZooKeeper is a high-performance coordination service for distributed
applications. It exposes common services - such as naming, configuration
management, synchronization, and group services - in a simple interface
so you don't have to write them from scratch. You can use it
off-the-shelf to implement consensus, group management, leader election,
and presence protocols. And you can build on it for your own, specific
needs.
Key features of the 3.1.0 release:
* Quota support
* BookKeeper - a system to reliably log streams of records
* JMX for server management
* many fixes, improvements, improved documentation, etc...
A bit about BookKeeper: a system to reliably log streams of records. In
BookKeeper, servers are "bookies", log streams are "ledgers", and each
unit of a log (aka record) is a "ledger entry". BookKeeper is designed
to be reliable; bookies, the servers that store ledgers can be
byzantine, which means that some subset of the bookies can fail, corrupt
data, discard data, but as long as there are enough correctly behaving
servers the service as a whole behaves correctly; the meta data for
BookKeeper is stored in ZooKeeper.
For ZooKeeper release details and downloads, visit:
http://hadoop.apache.org/zookeeper/releases.html
ZooKeeper 3.1.0 Release Notes are at:
http://hadoop.apache.org/zookeeper/docs/r3.1.0/releasenotes.html
Regards,
The ZooKeeper Team
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Also, you may consider checking a graph that we posted comparing the
performance of BookKeeper with the one of HDFS using a local file
system and local+NFS in the jira issue 5189 (https://issues.apache.org/jira/browse/HADOOP-5189
).
-Flavio
On Feb 20, 2009, at 10:05 AM, Flavio Junqueira wrote:
> Hi Bill, I'm sorry, I missed this message initially. I'm sending
> below a table that gives you throughput figures for BookKeeper. The
> rows correspond to distinct BookKeeper configuration (ensemble size,
> quorum size, entry type), and the columns to different values for
> the length of an entry in bytes. The throughput values correspond to
> one client writing 400K records (we call them entries)
> asynchronously to a ledger. Finally, the table shows write
> throughput in thousands of operations per second.
>
>
> 128 1024 8192
> 3-2-V 32.80 26.45 5.89
> 4-2-V 41.72 31.53 6.55
> 5-2-V 46.89 32.45 6.61
>
> 4-3-G 28.02 21.61 4.37
> 5-3-G 34.91 28.22 4.60
> 6-3-G 41.22 31.70 4.55
>
>
> Let me know if you have more questions, I appreciate your interest.
>
> Thanks,
> -Flavio
>
>
>
>
>
> On Feb 14, 2009, at 2:56 PM, Bill de hOra wrote:
>
>> Patrick Hunt wrote:
>>
>>> A bit about BookKeeper: a system to reliably log streams of
>>> records. In BookKeeper, servers are "bookies", log streams are
>>> "ledgers", and each unit of a log (aka record) is a "ledger
>>> entry". BookKeeper is designed to be reliable; bookies, the
>>> servers that store ledgers can be byzantine, which means that some
>>> subset of the bookies can fail, corrupt data, discard data, but as
>>> long as there are enough correctly behaving servers the service as
>>> a whole behaves correctly; the meta data for BookKeeper is stored
>>> in ZooKeeper.
>>
>> Hi Patrick,
>>
>> this sounds cool. Are there any figures on throughput, ie how many
>> records BookKeeper can process per second?
>>
>> Bill
>
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Also, you may consider checking a graph that we posted comparing the
performance of BookKeeper with the one of HDFS using a local file
system and local+NFS in the jira issue 5189 (https://issues.apache.org/jira/browse/HADOOP-5189
).
-Flavio
On Feb 20, 2009, at 10:05 AM, Flavio Junqueira wrote:
> Hi Bill, I'm sorry, I missed this message initially. I'm sending
> below a table that gives you throughput figures for BookKeeper. The
> rows correspond to distinct BookKeeper configuration (ensemble size,
> quorum size, entry type), and the columns to different values for
> the length of an entry in bytes. The throughput values correspond to
> one client writing 400K records (we call them entries)
> asynchronously to a ledger. Finally, the table shows write
> throughput in thousands of operations per second.
>
>
> 128 1024 8192
> 3-2-V 32.80 26.45 5.89
> 4-2-V 41.72 31.53 6.55
> 5-2-V 46.89 32.45 6.61
>
> 4-3-G 28.02 21.61 4.37
> 5-3-G 34.91 28.22 4.60
> 6-3-G 41.22 31.70 4.55
>
>
> Let me know if you have more questions, I appreciate your interest.
>
> Thanks,
> -Flavio
>
>
>
>
>
> On Feb 14, 2009, at 2:56 PM, Bill de hOra wrote:
>
>> Patrick Hunt wrote:
>>
>>> A bit about BookKeeper: a system to reliably log streams of
>>> records. In BookKeeper, servers are "bookies", log streams are
>>> "ledgers", and each unit of a log (aka record) is a "ledger
>>> entry". BookKeeper is designed to be reliable; bookies, the
>>> servers that store ledgers can be byzantine, which means that some
>>> subset of the bookies can fail, corrupt data, discard data, but as
>>> long as there are enough correctly behaving servers the service as
>>> a whole behaves correctly; the meta data for BookKeeper is stored
>>> in ZooKeeper.
>>
>> Hi Patrick,
>>
>> this sounds cool. Are there any figures on throughput, ie how many
>> records BookKeeper can process per second?
>>
>> Bill
>
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Hi Bill, I'm sorry, I missed this message initially. I'm sending below
a table that gives you throughput figures for BookKeeper. The rows
correspond to distinct BookKeeper configuration (ensemble size, quorum
size, entry type), and the columns to different values for the length
of an entry in bytes. The throughput values correspond to one client
writing 400K records (we call them entries) asynchronously to a
ledger. Finally, the table shows write throughput in thousands of
operations per second.
128 1024 8192
3-2-V 32.80 26.45 5.89
4-2-V 41.72 31.53 6.55
5-2-V 46.89 32.45 6.61
4-3-G 28.02 21.61 4.37
5-3-G 34.91 28.22 4.60
6-3-G 41.22 31.70 4.55
Let me know if you have more questions, I appreciate your interest.
Thanks,
-Flavio
On Feb 14, 2009, at 2:56 PM, Bill de hOra wrote:
> Patrick Hunt wrote:
>
>> A bit about BookKeeper: a system to reliably log streams of
>> records. In BookKeeper, servers are "bookies", log streams are
>> "ledgers", and each unit of a log (aka record) is a "ledger entry".
>> BookKeeper is designed to be reliable; bookies, the servers that
>> store ledgers can be byzantine, which means that some subset of the
>> bookies can fail, corrupt data, discard data, but as long as there
>> are enough correctly behaving servers the service as a whole
>> behaves correctly; the meta data for BookKeeper is stored in
>> ZooKeeper.
>
> Hi Patrick,
>
> this sounds cool. Are there any figures on throughput, ie how many
> records BookKeeper can process per second?
>
> Bill
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Stephen Corona <sc...@adknowledge.com>.
Is this similar to facebook's scribe server project? How "production-
ready" is bookkeeper?
Steve
On Feb 14, 2009, at 8:56 AM, "Bill de hOra" <bi...@dehora.net> wrote:
> Patrick Hunt wrote:
>
>> A bit about BookKeeper: a system to reliably log streams of
>> records. In
>> BookKeeper, servers are "bookies", log streams are "ledgers", and
>> each
>> unit of a log (aka record) is a "ledger entry". BookKeeper is
>> designed
>> to be reliable; bookies, the servers that store ledgers can be
>> byzantine, which means that some subset of the bookies can fail,
>> corrupt
>> data, discard data, but as long as there are enough correctly
>> behaving
>> servers the service as a whole behaves correctly; the meta data for
>> BookKeeper is stored in ZooKeeper.
>
> Hi Patrick,
>
> this sounds cool. Are there any figures on throughput, ie how many
> records BookKeeper can process per second?
>
> Bill
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Hi Bill, I'm sorry, I missed this message initially. I'm sending below
a table that gives you throughput figures for BookKeeper. The rows
correspond to distinct BookKeeper configuration (ensemble size, quorum
size, entry type), and the columns to different values for the length
of an entry in bytes. The throughput values correspond to one client
writing 400K records (we call them entries) asynchronously to a
ledger. Finally, the table shows write throughput in thousands of
operations per second.
128 1024 8192
3-2-V 32.80 26.45 5.89
4-2-V 41.72 31.53 6.55
5-2-V 46.89 32.45 6.61
4-3-G 28.02 21.61 4.37
5-3-G 34.91 28.22 4.60
6-3-G 41.22 31.70 4.55
Let me know if you have more questions, I appreciate your interest.
Thanks,
-Flavio
On Feb 14, 2009, at 2:56 PM, Bill de hOra wrote:
> Patrick Hunt wrote:
>
>> A bit about BookKeeper: a system to reliably log streams of
>> records. In BookKeeper, servers are "bookies", log streams are
>> "ledgers", and each unit of a log (aka record) is a "ledger entry".
>> BookKeeper is designed to be reliable; bookies, the servers that
>> store ledgers can be byzantine, which means that some subset of the
>> bookies can fail, corrupt data, discard data, but as long as there
>> are enough correctly behaving servers the service as a whole
>> behaves correctly; the meta data for BookKeeper is stored in
>> ZooKeeper.
>
> Hi Patrick,
>
> this sounds cool. Are there any figures on throughput, ie how many
> records BookKeeper can process per second?
>
> Bill
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Bill de hOra <bi...@dehora.net>.
Patrick Hunt wrote:
> A bit about BookKeeper: a system to reliably log streams of records. In
> BookKeeper, servers are "bookies", log streams are "ledgers", and each
> unit of a log (aka record) is a "ledger entry". BookKeeper is designed
> to be reliable; bookies, the servers that store ledgers can be
> byzantine, which means that some subset of the bookies can fail, corrupt
> data, discard data, but as long as there are enough correctly behaving
> servers the service as a whole behaves correctly; the meta data for
> BookKeeper is stored in ZooKeeper.
Hi Patrick,
this sounds cool. Are there any figures on throughput, ie how many
records BookKeeper can process per second?
Bill
Re: [ANNOUNCE] Apache ZooKeeper 3.1.0
Posted by Bill de hOra <bi...@dehora.net>.
Patrick Hunt wrote:
> A bit about BookKeeper: a system to reliably log streams of records. In
> BookKeeper, servers are "bookies", log streams are "ledgers", and each
> unit of a log (aka record) is a "ledger entry". BookKeeper is designed
> to be reliable; bookies, the servers that store ledgers can be
> byzantine, which means that some subset of the bookies can fail, corrupt
> data, discard data, but as long as there are enough correctly behaving
> servers the service as a whole behaves correctly; the meta data for
> BookKeeper is stored in ZooKeeper.
Hi Patrick,
this sounds cool. Are there any figures on throughput, ie how many
records BookKeeper can process per second?
Bill