Filtered Replication Question

[bsquared <bw...@gmail.com>]

Hello,

I am wondering how to implement a filtered replication with a blacklist
of sorts.  


(C..?) <--> (B?) <--> (A)

Where (A) is a local couchapp and (B) is an optional duplicate on the
Internet.  A and B replicate with one or more document stores (C..Z).  I
want to be able to ignore selected documents from the document
stores. The net affect is that user of (A) is deleting the document from
their replicated copies, but it still exists on the original document
store.

The idea that comes to mind for me is to have a design doc hold some
document metadata that I could use in a filter.

         my_doc_metadata : [ 
                         {id: doc_1, state:hidden
                         {id: doc_2, state:pinned}, ...]  

This doc would be replicated between (A) and (B) only.

Is this possible?  If so how could this be implemented in a filter?

Any alternatives come to mind?

Thank you.

-- 
Regards,
Brian

Re: Filtered Replication Question

[Mark Hahn <ma...@hahnca.com>]

You will have to have the "pinned" field in each doc.  Then you can easily
use that in a replication filter.  You cannot access other docs when
filtering.

On Sun, Jun 17, 2012 at 3:31 PM, bsquared <bw...@gmail.com> wrote:

> Hello,
>
> I am wondering how to implement a filtered replication with a blacklist
> of sorts.
>
>
> (C..?) <--> (B?) <--> (A)
>
> Where (A) is a local couchapp and (B) is an optional duplicate on the
> Internet.  A and B replicate with one or more document stores (C..Z).  I
> want to be able to ignore selected documents from the document
> stores. The net affect is that user of (A) is deleting the document from
> their replicated copies, but it still exists on the original document
> store.
>
> The idea that comes to mind for me is to have a design doc hold some
> document metadata that I could use in a filter.
>
>         my_doc_metadata : [
>                         {id: doc_1, state:hidden
>                         {id: doc_2, state:pinned}, ...]
>
> This doc would be replicated between (A) and (B) only.
>
> Is this possible?  If so how could this be implemented in a filter?
>
> Any alternatives come to mind?
>
> Thank you.
>
> --
> Regards,
> Brian
>
>

Re: Filtered Replication Question

[bsquared <bw...@gmail.com>]

Thank you for your insight.  Probably saved me from many headaches.
-- 
Regards,
Brian

Re: Filtered Replication Question

[Dave Cottlehuber <da...@muse.net.nz>]

On 18 June 2012 00:31, bsquared <bw...@gmail.com> wrote:
> Hello,
>
> I am wondering how to implement a filtered replication with a blacklist
> of sorts.
>
>
> (C..?) <--> (B?) <--> (A)
>
> Where (A) is a local couchapp and (B) is an optional duplicate on the
> Internet.  A and B replicate with one or more document stores (C..Z).  I
> want to be able to ignore selected documents from the document
> stores. The net affect is that user of (A) is deleting the document from
> their replicated copies, but it still exists on the original document
> store.
>
> The idea that comes to mind for me is to have a design doc hold some
> document metadata that I could use in a filter.
>
>         my_doc_metadata : [
>                         {id: doc_1, state:hidden
>                         {id: doc_2, state:pinned}, ...]
>
> This doc would be replicated between (A) and (B) only.
>
> Is this possible?  If so how could this be implemented in a filter?
>
> Any alternatives come to mind?
>
> Thank you.
>
> --
> Regards,
> Brian
>

As Mark says, replication and many other operations in CouchDB
are per-doc only by design.

You should assume that eventually all documents will end up
replicating to all nodes, and plan filters/views/replication/changes
feeds with that in mind. Even if you think you can avoid it, still
plan for the situation when it *does* happen. Experience shows
this is the better strategy.

Having said that;

You can always replicate selected documents with filtered replication
as Mark proposes, or pass an array of keys of docs you want.

Therefore its possible but not necessarily efficient to store a blacklist
document or views listing those ids, retrieve that and then only replicate
the remainder of (all_docs ids - blacklist). Whether this is efficient or
not will largely depend on bandwidth and the number of documents in
each set.

Finally, if the state of a document might change from pinned/hidden
then neither my nor Mark's approach will be sufficient to "delete" docs
on node B after their state has changed on node A.

A+
Dave