Why couchDB does full replication instead of filtered?

[Vassilis Radis <ra...@gmail.com>]

I am having this problem with couchdb (1.2.0) replication: I am using the
replicator db with this setup document:

    {
       "_id": "source_to_target",
       "_rev": "1-2a6510e28c2cc7caf0d58a85d705d2b8",
       "source": "http://xxxx:xxxx@localhost:5984/sourcedb",
       "target": "targetdb",
       "create_target": true,
       "continuous": true,
       "filter": "sourcedb/repl_filter",
       "query_params": {
           "someproperty": "somevalue"
       },
       "user_ctx": {
           "name": "someadmin",
           "roles": [
               "_admin"
           ]
       }
    }

The problem is that the filter function gets ignored. I am sure it is
correct because if I invoke:

    curl -X GET "
http://localhost:5984/sourcedb/_changes?filter=sourcedb/repl_filter&someproperty=somevalue&feed=continuous&style=all_docs&since=0
"

the results are correctly filtered.

I perform the test as follows:

I erase the target db completely. Then I erase the above replication and
recreate it from scratch.
As soon as the replication document shown above gets commited, the
replication starts and instead of the filtered docs, the target db gets ALL
the docs instead of the filtered, . The logs give a hint as to why:

    [Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] Replication
records differ. Scanning histories to find a common ancestor.
    [Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] no common ancestry
-- performing full replication

, which makes me scratch my head,because since I completely erase the
target db before starting the replication, why search for common ancsestors
with a not existent db? Apparently I am missing something, but I cant
figure it out. Any suggestions? Is this desired behaviour?

Re: Why couchDB does full replication instead of filtered?

[Vassilis Radis <ra...@gmail.com>]

Ok, here is some update

I tried another fresh target database with a new name. It works. So this
made me search a bit better in the logs and i found that although i was
erasing the replication document in each test, somewhere among those
deletions one of them failed to delete its underlying replication process
which continued to be up and running. So this zombie one, made me think
that the new ones didnt work. In fact they worked, filtering all the docs
out, but the zombie replication kept going on doing full replication.

It was difficult to distinguish in the logs that in fact there always were
2 replications running because the zombie performed full replication and
appended a lot to the logs, where the normal one didnt append almost
anything because it filters out all the docs, making me thing there is just
one, that ignores the filter function. I should have looked more carefully.

Anyway, since I always started the replications in futon by
erasing/inserting the replication doc and I never started a replication
from command-line or any other way, I assume deleting the replication
document is not guaranteed to cancel the underlying replication process.
Is this correct?

On Sat, Jul 21, 2012 at 2:53 AM, Vassilis Radis <ra...@gmail.com> wrote:

> I am having this problem with couchdb (1.2.0) replication: I am using the
> replicator db with this setup document:
>
>     {
>        "_id": "source_to_target",
>        "_rev": "1-2a6510e28c2cc7caf0d58a85d705d2b8",
>        "source": "http://xxxx:xxxx@localhost:5984/sourcedb",
>        "target": "targetdb",
>        "create_target": true,
>        "continuous": true,
>        "filter": "sourcedb/repl_filter",
>        "query_params": {
>            "someproperty": "somevalue"
>        },
>        "user_ctx": {
>            "name": "someadmin",
>            "roles": [
>                "_admin"
>            ]
>        }
>     }
>
> The problem is that the filter function gets ignored. I am sure it is
> correct because if I invoke:
>
>     curl -X GET "
> http://localhost:5984/sourcedb/_changes?filter=sourcedb/repl_filter&someproperty=somevalue&feed=continuous&style=all_docs&since=0
> "
>
> the results are correctly filtered.
>
> I perform the test as follows:
>
> I erase the target db completely. Then I erase the above replication and
> recreate it from scratch.
> As soon as the replication document shown above gets commited, the
> replication starts and instead of the filtered docs, the target db gets ALL
> the docs instead of the filtered, . The logs give a hint as to why:
>
>     [Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] Replication
> records differ. Scanning histories to find a common ancestor.
>     [Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] no common
> ancestry -- performing full replication
>
> , which makes me scratch my head,because since I completely erase the
> target db before starting the replication, why search for common ancsestors
> with a not existent db? Apparently I am missing something, but I cant
> figure it out. Any suggestions? Is this desired behaviour?
>