You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Andreas Kemkes <a5...@yahoo.com> on 2012/07/12 01:43:46 UTC
Filtered replication topologies
To speed up the process of breaking up one monolithic database into multiples, I used an intermediate database to get to the end result:
M -- pre-filtered replication --> I -- 1..n filtered replications --> R1..n
Afterwards I tried to set up continuous filtered replications directly between M and R1..n. It seems that the replication algorithm runs through the entire file again and does not take into account the last entry in Rx.
To make proper use of the existing state of the databases R1..n, does the continuous replication have the same source (i.e., I instead of M)?
Is it possible to provide a topology agnostic implementation?
What would be my best course of action?
-- Andreas
Re: Filtered replication topologies
Posted by Andreas Kemkes <a5...@yahoo.com>.
Mathias:
That is excellent advise. Still learning feature by feature.
Thanks,
Andreas
________________________________
From: Mathias Leppich <ml...@muhqu.de>
To: user@couchdb.apache.org; Andreas Kemkes <a5...@yahoo.com>
Sent: Thursday, July 12, 2012 12:22 AM
Subject: Re: Filtered replication topologies
Hi Andreas,
If you replicated to R1..n via an intermediate database they will not know the source DB.
But if you KNOW that M's seq N has already made its way via I to the Rs, you can use that seq N as a start seq parameter for a new replication between M and the Rs.
e.g.
curl -sg -X POST 'http://sourceHost:5984/_replicate' -H 'Content-Type:application/json' -d '{
"source":"sourceDB",
"target":"http://targetHost:5984/targetDB",
"filter":"filters_erl/no_design",
"continuous":true,
"since_seq":59210797
}'
- mathias
On Jul 12, 2012, at 1:43 , Andreas Kemkes wrote:
> To speed up the process of breaking up one monolithic database into multiples, I used an intermediate database to get to the end result:
>
>
>
> M -- pre-filtered replication --> I -- 1..n filtered replications --> R1..n
>
>
> Afterwards I tried to set up continuous filtered replications directly between M and R1..n. It seems that the replication algorithm runs through the entire file again and does not take into account the last entry in Rx.
>
> To make proper use of the existing state of the databases R1..n, does the continuous replication have the same source (i.e., I instead of M)?
>
> Is it possible to provide a topology agnostic implementation?
>
> What would be my best course of action?
>
> -- Andreas
Re: Filtered replication topologies
Posted by Mathias Leppich <ml...@muhqu.de>.
Hi Andreas,
If you replicated to R1..n via an intermediate database they will not know the source DB.
But if you KNOW that M's seq N has already made its way via I to the Rs, you can use that seq N as a start seq parameter for a new replication between M and the Rs.
e.g.
curl -sg -X POST 'http://sourceHost:5984/_replicate' -H 'Content-Type:application/json' -d '{
"source":"sourceDB",
"target":"http://targetHost:5984/targetDB",
"filter":"filters_erl/no_design",
"continuous":true,
"since_seq":59210797
}'
- mathias
On Jul 12, 2012, at 1:43 , Andreas Kemkes wrote:
> To speed up the process of breaking up one monolithic database into multiples, I used an intermediate database to get to the end result:
>
>
>
> M -- pre-filtered replication --> I -- 1..n filtered replications --> R1..n
>
>
> Afterwards I tried to set up continuous filtered replications directly between M and R1..n. It seems that the replication algorithm runs through the entire file again and does not take into account the last entry in Rx.
>
> To make proper use of the existing state of the databases R1..n, does the continuous replication have the same source (i.e., I instead of M)?
>
> Is it possible to provide a topology agnostic implementation?
>
> What would be my best course of action?
>
> -- Andreas