You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Robert Elliot <ro...@lidalia.org.uk> on 2011/08/19 15:32:01 UTC
Multi node topology
Hi,
We are considering setting up a cluster with more than two nodes. There is a lot of documentation regarding two nodes but we couldn't find an exact answer for let's say a cluster of 4 nodes.
Would you recommend a multi-master setup where all nodes receive writes? This would be simpler to setup and administer, and would also be the most fault tolerant (any combination of nodes can be shutdown so long as one is still active).
If so, should we use only push replication? Or only pull replication? Or a combination of both?
Assuming we are using pull replication within 4 nodes: A, B, C and D. Should we set up A to pull changes from B, C and D, B to pull changes from A, C and D, C to pull changes from A, B and D and D to pull changes from A, B and C? Is this the recommended approach?
Thanks for any guidance,
Rob
Re: Multi node topology
Posted by Robert Elliot <ro...@lidalia.org.uk>.
That's very useful to know - thanks. We'll give that some further thought. Is there some documentation we've missed on the pros/cons/pitfalls of multi-master replication?
Rob
----- Original Message -----
> From: "Mark J. Reed" <ma...@gmail.com>
> To: user@couchdb.apache.org
> Sent: Monday, 22 August, 2011 12:20:09 PM
> Subject: Re: Multi node topology
> Load balancing is not sufficient for all uses, in particular if you
> use the
> changes APi. Different nodes have different sequence numbers, so if
> you
> subscribe to the changes feed, lose the connection, and get a
> different node
> on reconnect, you will lose track of your place, and either miss
> changes or
> reprocess some.
>
> On Monday, August 22, 2011, Robert Elliot <ro...@lidalia.org.uk> wrote:
> > No probs - we've got load balancers already available that we're
> > familiar
> with scripting and administering, so if the cluster is not partitioned
> treating it as a single node is as easy for us just using our load
> balancers.
> >
> > Rob
> >
>
> --
> Mark J. Reed <ma...@gmail.com>
Re: Multi node topology
Posted by "Mark J. Reed" <ma...@gmail.com>.
Load balancing is not sufficient for all uses, in particular if you use the
changes APi. Different nodes have different sequence numbers, so if you
subscribe to the changes feed, lose the connection, and get a different node
on reconnect, you will lose track of your place, and either miss changes or
reprocess some.
On Monday, August 22, 2011, Robert Elliot <ro...@lidalia.org.uk> wrote:
> No probs - we've got load balancers already available that we're familiar
with scripting and administering, so if the cluster is not partitioned
treating it as a single node is as easy for us just using our load
balancers.
>
> Rob
>
> ----- Original Message -----
>> From: "Paul Davis" <pa...@gmail.com>
>> To: user@couchdb.apache.org
>> Sent: Friday, 19 August, 2011 7:37:04 PM
>> Subject: Re: Multi node topology
>> Robert,
>>
>> Sorry for the confusion. I jumped to the conclusion that you'd want to
>> be looking at the four nodes as a single logical node which is what
>> those projects provide even beyond the sharding and partitioning.
>>
>> On Aug 19, 2011, at 1:21 PM, Robert Elliot wrote:
>>
>> > Thanks; we're aware of both solutions, but as we do not currently
>> > need to partition/shard we are interested in exploring what CouchDB
>> > could do by itself, since it supports bi-directional replication.
>> >
>> > Is the answer "the CouchDB developers recommend not running CouchDB
>> > in a multi-master replicated topology"?
>> >
>> > On 19 Aug 2011, at 18:26, Paul Davis wrote:
>> >
>> >> Robert,
>> >>
>> >> There are two projects for running CouchDB in a cluster:
>> >> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
>> >> nginx module and Python proxy that handles managing multiple
>> >> backend
>> >> CouchDB nodes. BigCouch is a pure Erlang implementation that works
>> >> more like Dynamo.
>> >>
>> >> [1] http://tilgovi.github.com/couchdb-lounge/
>> >> [2] http://github.com/cloudant/bigcouch
>> >>
>> >> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk>
>> >> wrote:
>> >>> Hi,
>> >>>
>> >>> We are considering setting up a cluster with more than two nodes.
>> >>> There is a lot of documentation regarding two nodes but we
>> >>> couldn't find an exact answer for let's say a cluster of 4 nodes.
>> >>>
>> >>> Would you recommend a multi-master setup where all nodes receive
>> >>> writes? This would be simpler to setup and administer, and would
>> >>> also be the most fault tolerant (any combination of nodes can be
>> >>> shutdown so long as one is still active).
>> >>>
>> >>> If so, should we use only push replication? Or only pull
>> >>> replication? Or a combination of both?
>> >>>
>> >>> Assuming we are using pull replication within 4 nodes: A, B, C and
>> >>> D. Should we set up A to pull changes from B, C and D, B to pull
>> >>> changes from A, C and D, C to pull changes from A, B and D and D
>> >>> to pull changes from A, B and C? Is this the recommended approach?
>> >>>
>> >>> Thanks for any guidance,
>> >>>
>> >>> Rob
>> >>>
>> >
>
--
Mark J. Reed <ma...@gmail.com>
Re: Multi node topology
Posted by Robert Elliot <ro...@lidalia.org.uk>.
No probs - we've got load balancers already available that we're familiar with scripting and administering, so if the cluster is not partitioned treating it as a single node is as easy for us just using our load balancers.
Rob
----- Original Message -----
> From: "Paul Davis" <pa...@gmail.com>
> To: user@couchdb.apache.org
> Sent: Friday, 19 August, 2011 7:37:04 PM
> Subject: Re: Multi node topology
> Robert,
>
> Sorry for the confusion. I jumped to the conclusion that you'd want to
> be looking at the four nodes as a single logical node which is what
> those projects provide even beyond the sharding and partitioning.
>
> On Aug 19, 2011, at 1:21 PM, Robert Elliot wrote:
>
> > Thanks; we're aware of both solutions, but as we do not currently
> > need to partition/shard we are interested in exploring what CouchDB
> > could do by itself, since it supports bi-directional replication.
> >
> > Is the answer "the CouchDB developers recommend not running CouchDB
> > in a multi-master replicated topology"?
> >
> > On 19 Aug 2011, at 18:26, Paul Davis wrote:
> >
> >> Robert,
> >>
> >> There are two projects for running CouchDB in a cluster:
> >> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
> >> nginx module and Python proxy that handles managing multiple
> >> backend
> >> CouchDB nodes. BigCouch is a pure Erlang implementation that works
> >> more like Dynamo.
> >>
> >> [1] http://tilgovi.github.com/couchdb-lounge/
> >> [2] http://github.com/cloudant/bigcouch
> >>
> >> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk>
> >> wrote:
> >>> Hi,
> >>>
> >>> We are considering setting up a cluster with more than two nodes.
> >>> There is a lot of documentation regarding two nodes but we
> >>> couldn't find an exact answer for let's say a cluster of 4 nodes.
> >>>
> >>> Would you recommend a multi-master setup where all nodes receive
> >>> writes? This would be simpler to setup and administer, and would
> >>> also be the most fault tolerant (any combination of nodes can be
> >>> shutdown so long as one is still active).
> >>>
> >>> If so, should we use only push replication? Or only pull
> >>> replication? Or a combination of both?
> >>>
> >>> Assuming we are using pull replication within 4 nodes: A, B, C and
> >>> D. Should we set up A to pull changes from B, C and D, B to pull
> >>> changes from A, C and D, C to pull changes from A, B and D and D
> >>> to pull changes from A, B and C? Is this the recommended approach?
> >>>
> >>> Thanks for any guidance,
> >>>
> >>> Rob
> >>>
> >
Re: Multi node topology
Posted by Paul Davis <pa...@gmail.com>.
Robert,
Sorry for the confusion. I jumped to the conclusion that you'd want to be looking at the four nodes as a single logical node which is what those projects provide even beyond the sharding and partitioning.
On Aug 19, 2011, at 1:21 PM, Robert Elliot wrote:
> Thanks; we're aware of both solutions, but as we do not currently need to partition/shard we are interested in exploring what CouchDB could do by itself, since it supports bi-directional replication.
>
> Is the answer "the CouchDB developers recommend not running CouchDB in a multi-master replicated topology"?
>
> On 19 Aug 2011, at 18:26, Paul Davis wrote:
>
>> Robert,
>>
>> There are two projects for running CouchDB in a cluster:
>> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
>> nginx module and Python proxy that handles managing multiple backend
>> CouchDB nodes. BigCouch is a pure Erlang implementation that works
>> more like Dynamo.
>>
>> [1] http://tilgovi.github.com/couchdb-lounge/
>> [2] http://github.com/cloudant/bigcouch
>>
>> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk> wrote:
>>> Hi,
>>>
>>> We are considering setting up a cluster with more than two nodes. There is a lot of documentation regarding two nodes but we couldn't find an exact answer for let's say a cluster of 4 nodes.
>>>
>>> Would you recommend a multi-master setup where all nodes receive writes? This would be simpler to setup and administer, and would also be the most fault tolerant (any combination of nodes can be shutdown so long as one is still active).
>>>
>>> If so, should we use only push replication? Or only pull replication? Or a combination of both?
>>>
>>> Assuming we are using pull replication within 4 nodes: A, B, C and D. Should we set up A to pull changes from B, C and D, B to pull changes from A, C and D, C to pull changes from A, B and D and D to pull changes from A, B and C? Is this the recommended approach?
>>>
>>> Thanks for any guidance,
>>>
>>> Rob
>>>
>
Re: Multi node topology
Posted by Robert Elliot <ro...@lidalia.org.uk>.
OK, great. So my next question is what we need to do to restore after reboting a node. We know that the replication information is not persisted. So if in the scenario I outlined we restart A, obviously we need to tell it to pull from B, C and D again. However, do we also need to tell B, C and D to pull from A? Or will they have been trying to do that anyway? Should we really have told B, C and D to *stop* pulling from A before we stopped A in a deliberate restart scenario?
Thanks,
Rob
----- Original Message -----
> From: "Adam Kocoloski" <ko...@apache.org>
> To: user@couchdb.apache.org
> Sent: Friday, 19 August, 2011 7:29:23 PM
> Subject: Re: Multi node topology
> Hi Robert, no, I wouldn't say that. The projects Paul mentioned can
> simplify some of the setup of a multi-master deployment even if you
> don't take advantage of the sharding, but they're by no means required
> for it.
>
> To answer your questions, push and pull replications are approximately
> equal in their efficiency these days. The approach you outlined should
> be fine for a cluster of 4 nodes. Cheers,
>
> Adam
>
> On Aug 19, 2011, at 2:21 PM, Robert Elliot wrote:
>
> > Thanks; we're aware of both solutions, but as we do not currently
> > need to partition/shard we are interested in exploring what CouchDB
> > could do by itself, since it supports bi-directional replication.
> >
> > Is the answer "the CouchDB developers recommend not running CouchDB
> > in a multi-master replicated topology"?
> >
> > On 19 Aug 2011, at 18:26, Paul Davis wrote:
> >
> >> Robert,
> >>
> >> There are two projects for running CouchDB in a cluster:
> >> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
> >> nginx module and Python proxy that handles managing multiple
> >> backend
> >> CouchDB nodes. BigCouch is a pure Erlang implementation that works
> >> more like Dynamo.
> >>
> >> [1] http://tilgovi.github.com/couchdb-lounge/
> >> [2] http://github.com/cloudant/bigcouch
> >>
> >> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk>
> >> wrote:
> >>> Hi,
> >>>
> >>> We are considering setting up a cluster with more than two nodes.
> >>> There is a lot of documentation regarding two nodes but we
> >>> couldn't find an exact answer for let's say a cluster of 4 nodes.
> >>>
> >>> Would you recommend a multi-master setup where all nodes receive
> >>> writes? This would be simpler to setup and administer, and would
> >>> also be the most fault tolerant (any combination of nodes can be
> >>> shutdown so long as one is still active).
> >>>
> >>> If so, should we use only push replication? Or only pull
> >>> replication? Or a combination of both?
> >>>
> >>> Assuming we are using pull replication within 4 nodes: A, B, C and
> >>> D. Should we set up A to pull changes from B, C and D, B to pull
> >>> changes from A, C and D, C to pull changes from A, B and D and D
> >>> to pull changes from A, B and C? Is this the recommended approach?
> >>>
> >>> Thanks for any guidance,
> >>>
> >>> Rob
> >>>
> >
Re: Multi node topology
Posted by Adam Kocoloski <ko...@apache.org>.
Hi Robert, no, I wouldn't say that. The projects Paul mentioned can simplify some of the setup of a multi-master deployment even if you don't take advantage of the sharding, but they're by no means required for it.
To answer your questions, push and pull replications are approximately equal in their efficiency these days. The approach you outlined should be fine for a cluster of 4 nodes. Cheers,
Adam
On Aug 19, 2011, at 2:21 PM, Robert Elliot wrote:
> Thanks; we're aware of both solutions, but as we do not currently need to partition/shard we are interested in exploring what CouchDB could do by itself, since it supports bi-directional replication.
>
> Is the answer "the CouchDB developers recommend not running CouchDB in a multi-master replicated topology"?
>
> On 19 Aug 2011, at 18:26, Paul Davis wrote:
>
>> Robert,
>>
>> There are two projects for running CouchDB in a cluster:
>> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
>> nginx module and Python proxy that handles managing multiple backend
>> CouchDB nodes. BigCouch is a pure Erlang implementation that works
>> more like Dynamo.
>>
>> [1] http://tilgovi.github.com/couchdb-lounge/
>> [2] http://github.com/cloudant/bigcouch
>>
>> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk> wrote:
>>> Hi,
>>>
>>> We are considering setting up a cluster with more than two nodes. There is a lot of documentation regarding two nodes but we couldn't find an exact answer for let's say a cluster of 4 nodes.
>>>
>>> Would you recommend a multi-master setup where all nodes receive writes? This would be simpler to setup and administer, and would also be the most fault tolerant (any combination of nodes can be shutdown so long as one is still active).
>>>
>>> If so, should we use only push replication? Or only pull replication? Or a combination of both?
>>>
>>> Assuming we are using pull replication within 4 nodes: A, B, C and D. Should we set up A to pull changes from B, C and D, B to pull changes from A, C and D, C to pull changes from A, B and D and D to pull changes from A, B and C? Is this the recommended approach?
>>>
>>> Thanks for any guidance,
>>>
>>> Rob
>>>
>
Re: Multi node topology
Posted by Robert Elliot <ro...@lidalia.org.uk>.
Thanks; we're aware of both solutions, but as we do not currently need to partition/shard we are interested in exploring what CouchDB could do by itself, since it supports bi-directional replication.
Is the answer "the CouchDB developers recommend not running CouchDB in a multi-master replicated topology"?
On 19 Aug 2011, at 18:26, Paul Davis wrote:
> Robert,
>
> There are two projects for running CouchDB in a cluster:
> CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
> nginx module and Python proxy that handles managing multiple backend
> CouchDB nodes. BigCouch is a pure Erlang implementation that works
> more like Dynamo.
>
> [1] http://tilgovi.github.com/couchdb-lounge/
> [2] http://github.com/cloudant/bigcouch
>
> On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk> wrote:
>> Hi,
>>
>> We are considering setting up a cluster with more than two nodes. There is a lot of documentation regarding two nodes but we couldn't find an exact answer for let's say a cluster of 4 nodes.
>>
>> Would you recommend a multi-master setup where all nodes receive writes? This would be simpler to setup and administer, and would also be the most fault tolerant (any combination of nodes can be shutdown so long as one is still active).
>>
>> If so, should we use only push replication? Or only pull replication? Or a combination of both?
>>
>> Assuming we are using pull replication within 4 nodes: A, B, C and D. Should we set up A to pull changes from B, C and D, B to pull changes from A, C and D, C to pull changes from A, B and D and D to pull changes from A, B and C? Is this the recommended approach?
>>
>> Thanks for any guidance,
>>
>> Rob
>>
Re: Multi node topology
Posted by Paul Davis <pa...@gmail.com>.
Robert,
There are two projects for running CouchDB in a cluster:
CouchDB-Lounge and BigCouch. CouchDB-Lounge is a combination of an
nginx module and Python proxy that handles managing multiple backend
CouchDB nodes. BigCouch is a pure Erlang implementation that works
more like Dynamo.
[1] http://tilgovi.github.com/couchdb-lounge/
[2] http://github.com/cloudant/bigcouch
On Fri, Aug 19, 2011 at 8:32 AM, Robert Elliot <ro...@lidalia.org.uk> wrote:
> Hi,
>
> We are considering setting up a cluster with more than two nodes. There is a lot of documentation regarding two nodes but we couldn't find an exact answer for let's say a cluster of 4 nodes.
>
> Would you recommend a multi-master setup where all nodes receive writes? This would be simpler to setup and administer, and would also be the most fault tolerant (any combination of nodes can be shutdown so long as one is still active).
>
> If so, should we use only push replication? Or only pull replication? Or a combination of both?
>
> Assuming we are using pull replication within 4 nodes: A, B, C and D. Should we set up A to pull changes from B, C and D, B to pull changes from A, C and D, C to pull changes from A, B and D and D to pull changes from A, B and C? Is this the recommended approach?
>
> Thanks for any guidance,
>
> Rob
>