A couple of noob questions.

[Tim Uckun <ti...@gmail.com>]

Has anybody done any benchmarks on the replication speed of couchdb?
If I have four servers each with couchdb running on them along with a
client application that talks to it and one client makes an entry how
long will it take for the other three to get the entry?

If each client makes a thousand entries in a 15 minute period will
there be a huge backlog built up waiting to replicate?

Also is it possible to put files in the couchdb? Specifically media
files like wav. Will they replicate as well?

Re: A couple of noob questions.

[Julian Moritz <ma...@julianmoritz.de>]

Hi,

Tim Uckun schrieb:
> Has anybody done any benchmarks on the replication speed of couchdb?
> If I have four servers each with couchdb running on them along with a
> client application that talks to it and one client makes an entry how
> long will it take for the other three to get the entry?
> 
> If each client makes a thousand entries in a 15 minute period will
> there be a huge backlog built up waiting to replicate?
> 
> Also is it possible to put files in the couchdb? Specifically media
> files like wav. Will they replicate as well?
> 

google keyword for mediafiles is "attachment".

Regards
Julian

Re: A couple of noob questions.

[Tim Uckun <ti...@gmail.com>]

>
> In the upcoming release CouchDB will have some facility for continuous
> replication, which takes the burden off of you to trigger on update and
> reduces the lag for updates to show up on the target.  Also, each DB will
> have a _changes feed that you can listen to.  A script that listens to
> _changes on both source and target could do a reasonable calculation of the
> replication lag.
>

Thank you for your detailed response.

Re: A couple of noob questions.

[Adam Kocoloski <ko...@apache.org>]

On Aug 12, 2009, at 7:52 AM, Tim Uckun wrote:

> Has anybody done any benchmarks on the replication speed of couchdb?
> If I have four servers each with couchdb running on them along with a
> client application that talks to it and one client makes an entry how
> long will it take for the other three to get the entry?

In CouchDB 0.9.x (0.9.1 is the current release) you have to trigger  
the replication manually.  You can write a script that listens for  
update_notifications and POSTs to _replicate.  So that's once source  
of lag.

Replication basically loops through the following 4 steps:

1. ask the source for new updates since the last replication
2. ask the target which of those updates it does not have
3. get the missing updates on the source
4. write them to the target

In most cases, 2 of these will be local requests and two will be  
remote HTTP requests, so the latency between the source and target  
enters into the equation twice.

Will all that said, its certainly possible to configure things so that  
latencies are O(100ms) for single document updates.  I'm afraid there  
isn't a great way to measure this in 0.9.x.

In the upcoming release CouchDB will have some facility for continuous  
replication, which takes the burden off of you to trigger on update  
and reduces the lag for updates to show up on the target.  Also, each  
DB will have a _changes feed that you can listen to.  A script that  
listens to _changes on both source and target could do a reasonable  
calculation of the replication lag.

> If each client makes a thousand entries in a 15 minute period will
> there be a huge backlog built up waiting to replicate?

It depends on how you're triggering the replication.  If you're  
triggering it on update (or using continuous replication in the  
upcoming release) there shouldn't be any backlog.

In CouchDB 0.9.x push replication (local source, remote target)  
generally seems to run at about 80-100 docs/second for me using O(1KB)  
documents and no attachments.  Pull replication (remote source, local  
target) is significantly slower because it replicates documents one at  
a time.  The trunk code is faster than either of these.

> Also is it possible to put files in the couchdb? Specifically media
> files like wav.

Yes, absolutely.  We do this using document attachments -- PUT /dbname/ 
docname/media.wav

> Will they replicate as well?

Yes, although replicating attachments uses a lot more memory than it  
should.  We're working on that.  I'd stuck to a couple of MBs or less  
per document for now.

Best, Adam