RE: [Zookeeper-user] Scalability and ZooKeeper java client thread safety

[Jacob Levy <jy...@yahoo-inc.com>]

Hi Anthony

Scalability depends on the mix of reads and writes you do with
ZooKeeper.

If all communication is going through ZooKeeper, then you're probably
expecting 1 or more reads per write. In that case (about equal numbers
of reads and writes), your scalability is limited by the number of
writes to ZooKeeper. Adding more quorum servers will likely *hurt*
performance, not help, because a majority needs to be in agreement about
the order of all writes.

If you're only going to use ZooKeeper for very occasional updates and
mostly read values from the znodes, then adding more quorum servers will
allow you to scale to more clients.

In practice we've found that the aggregate read throughput scales well
with the number of quorum servers, whereas the aggregate write
throughput scales inversely (i.e. declines) with the number of quorum
servers. The sweet spot where good read and write throughput is obtained
seems to be five quorum servers, with today's networking and hardware
technology.

--Jacob

-----Original Message-----
From: zookeeper-user-bounces@lists.sourceforge.net
[mailto:zookeeper-user-bounces@lists.sourceforge.net] On Behalf Of
Anthony Urso
Sent: Tuesday, July 15, 2008 3:13 PM
To: zookeeper-user@lists.sourceforge.net;
zookeeper-user@hadoop.apache.org
Subject: [Zookeeper-user] Scalability and ZooKeeper java client thread
safety

I have a distributed server cluster and each of the many threads on
each server has a ZooKeeper java client used to communicate to other
machines.  What kind of scalability can I expect from the zookeeper
server?  Can I just add more QuorumServers infinitely?

Also, assuming I can coordinate the default watcher behavior of these
various threads, would it be safe to share the ZooKeeper objects
between them?

------------------------------------------------------------------------
-
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Zookeeper-user mailing list
Zookeeper-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/zookeeper-user