You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Daniel Carr <d....@gmail.com> on 2013/09/05 00:21:56 UTC
_active_tasks replication progress stuck at 98
Hi all.
I'm using couchDB 1.2.1
I have a replication that I start by posting to _replicate. When I poll
_active_tasks the progress gets stuck on 98, and will not go up. This is
despite all the documents being replicated.
It appears that the source_seq is higher than it should be. As when I poll
the remotehost _changes feed, it gives a lower last_seq than the source_seq
in localhost's _active_tasks.
Any ideas what's going on here?
My code relies on the progress getting to 100, should I not do this?
Cheers,
Daniel
I've looked the entry in the log showing the start of the replication, but
can't find it. If it would be helpful, please tell me what to search for.
curl http://localhost:5984/_active_tasks:
[{
checkpointed_source_seq: 251
continuous: true
doc_id: null
doc_write_failures: 0
docs_read: 150
docs_written: 150
missing_revisions_found: 150
pid: "<0.502.0>"
progress: 98
replication_id: "afeb3a2e577aee71c767ed96090505c6+continuous"
revisions_checked: 150
source: "http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/"
source_seq: 255
started_on: 1378331698
target: "aml"
type: "replication"
updated_on: 1378331916
}]
curl
http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/_changes :
{"results":[
...
{"seq":251,"id":"fc47efd9448a47b990b01a412f7da8ce","changes":[{"rev":"1-65d41b82c27c6df7891a6bd875b18c1c"}]}
],
"last_seq":251}
Re: _active_tasks replication progress stuck at 98
Posted by Daniel Carr <d....@gmail.com>.
Hi B.
I've got a number of replications going this way and they definitely reach
100. The only time I have observed them not doing this was for a push
replication where there were permission issues (changed a document type
that is read only on the destination database).
The problem here seems to be that the source_seq is wrong.
My understanding is that 'progress' = 'checkpointed_source_seq' / 'source_seq'
* 100. So, it should reach 100 when it's caught up to the changes feed on
remotehost.
-Daniel
On Thu, Sep 5, 2013 at 8:24 AM, Robert Newson <rn...@apache.org> wrote:
> continuous replication, though. Not sure it will reach 100 (and it
> would obviously be meaningless if it did..).
>
> B.
>
>
> On 4 September 2013 23:21, Daniel Carr <d....@gmail.com> wrote:
> > Hi all.
> >
> > I'm using couchDB 1.2.1
> >
> > I have a replication that I start by posting to _replicate. When I poll
> > _active_tasks the progress gets stuck on 98, and will not go up. This is
> > despite all the documents being replicated.
> >
> > It appears that the source_seq is higher than it should be. As when I
> poll
> > the remotehost _changes feed, it gives a lower last_seq than the
> source_seq
> > in localhost's _active_tasks.
> >
> > Any ideas what's going on here?
> > My code relies on the progress getting to 100, should I not do this?
> >
> > Cheers,
> > Daniel
> >
> >
> > I've looked the entry in the log showing the start of the replication,
> but
> > can't find it. If it would be helpful, please tell me what to search for.
> >
> >
> > curl http://localhost:5984/_active_tasks:
> >
> > [{
> > checkpointed_source_seq: 251
> > continuous: true
> > doc_id: null
> > doc_write_failures: 0
> > docs_read: 150
> > docs_written: 150
> > missing_revisions_found: 150
> > pid: "<0.502.0>"
> > progress: 98
> > replication_id: "afeb3a2e577aee71c767ed96090505c6+continuous"
> > revisions_checked: 150
> > source: "
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/"
> > source_seq: 255
> > started_on: 1378331698
> > target: "aml"
> > type: "replication"
> > updated_on: 1378331916
> > }]
> >
> > curl
> >
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/_changes:
> >
> > {"results":[
> > ...
> >
> {"seq":251,"id":"fc47efd9448a47b990b01a412f7da8ce","changes":[{"rev":"1-65d41b82c27c6df7891a6bd875b18c1c"}]}
> > ],
> > "last_seq":251}
>
Re: _active_tasks replication progress stuck at 98
Posted by Robert Newson <rn...@apache.org>.
continuous replication, though. Not sure it will reach 100 (and it
would obviously be meaningless if it did..).
B.
On 4 September 2013 23:21, Daniel Carr <d....@gmail.com> wrote:
> Hi all.
>
> I'm using couchDB 1.2.1
>
> I have a replication that I start by posting to _replicate. When I poll
> _active_tasks the progress gets stuck on 98, and will not go up. This is
> despite all the documents being replicated.
>
> It appears that the source_seq is higher than it should be. As when I poll
> the remotehost _changes feed, it gives a lower last_seq than the source_seq
> in localhost's _active_tasks.
>
> Any ideas what's going on here?
> My code relies on the progress getting to 100, should I not do this?
>
> Cheers,
> Daniel
>
>
> I've looked the entry in the log showing the start of the replication, but
> can't find it. If it would be helpful, please tell me what to search for.
>
>
> curl http://localhost:5984/_active_tasks:
>
> [{
> checkpointed_source_seq: 251
> continuous: true
> doc_id: null
> doc_write_failures: 0
> docs_read: 150
> docs_written: 150
> missing_revisions_found: 150
> pid: "<0.502.0>"
> progress: 98
> replication_id: "afeb3a2e577aee71c767ed96090505c6+continuous"
> revisions_checked: 150
> source: "http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/"
> source_seq: 255
> started_on: 1378331698
> target: "aml"
> type: "replication"
> updated_on: 1378331916
> }]
>
> curl
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/_changes :
>
> {"results":[
> ...
> {"seq":251,"id":"fc47efd9448a47b990b01a412f7da8ce","changes":[{"rev":"1-65d41b82c27c6df7891a6bd875b18c1c"}]}
> ],
> "last_seq":251}
Re: _active_tasks replication progress stuck at 98
Posted by Daniel Carr <d....@gmail.com>.
Hi Simon.
There's 2 design documents. The account doing the replication has admin
privileges on it's instance of couchDB, but not on the machine it is
pulling from.
I think I see what you're getting at. I did the following things (in order)
1. Poll localhost/_active_tasks
checkpoint_source_seq: 251,
source_seq: 255
2. Poll remotehost/_changes
last_seq: 255
3. I modified a document (not a _design document) on remotehost.
4. Poll _active_tasks on localhost:
checkpoint_source_seq: 256,
source_seq: 256
5. Poll remotehost/_changes
last_seq: 256
So, what happened? seq 252-255 were modifications to a document that the
client does not have permission to read, and this caused the changes feed
to ignore it, thus returning a last_seq that was lower than the "true"
last_seq? However, the replication processes have the 'true' last_seq, and
_active_tasks somehow gets both of them?
If so, is this behaviour common knowledge, or desirable?
On Thu, Sep 5, 2013 at 3:52 PM, Simon Metson <si...@cloudant.com> wrote:
> Are there 4 design docs by any chance and does the account doing the
> replication have admin privileges?
>
>
> On Wednesday, 4 September 2013 at 23:21, Daniel Carr wrote:
>
> > Hi all.
> >
> > I'm using couchDB 1.2.1
> >
> > I have a replication that I start by posting to _replicate. When I poll
> > _active_tasks the progress gets stuck on 98, and will not go up. This is
> > despite all the documents being replicated.
> >
> > It appears that the source_seq is higher than it should be. As when I
> poll
> > the remotehost _changes feed, it gives a lower last_seq than the
> source_seq
> > in localhost's _active_tasks.
> >
> > Any ideas what's going on here?
> > My code relies on the progress getting to 100, should I not do this?
> >
> > Cheers,
> > Daniel
> >
> >
> > I've looked the entry in the log showing the start of the replication,
> but
> > can't find it. If it would be helpful, please tell me what to search for.
> >
> >
> > curl http://localhost:5984/_active_tasks:
> >
> > [{
> > checkpointed_source_seq: 251
> > continuous: true
> > doc_id: null
> > doc_write_failures: 0
> > docs_read: 150
> > docs_written: 150
> > missing_revisions_found: 150
> > pid: "<0.502.0>"
> > progress: 98
> > replication_id: "afeb3a2e577aee71c767ed96090505c6+continuous"
> > revisions_checked: 150
> > source: "
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/"
> > source_seq: 255
> > started_on: 1378331698
> > target: "aml"
> > type: "replication"
> > updated_on: 1378331916
> > }]
> >
> > curl
> >
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/_changes:
> >
> > {"results":[
> > ...
> >
> {"seq":251,"id":"fc47efd9448a47b990b01a412f7da8ce","changes":[{"rev":"1-65d41b82c27c6df7891a6bd875b18c1c"}]}
> > ],
> > "last_seq":251}
> >
> >
>
>
>
Re: _active_tasks replication progress stuck at 98
Posted by Simon Metson <si...@cloudant.com>.
Are there 4 design docs by any chance and does the account doing the replication have admin privileges?
On Wednesday, 4 September 2013 at 23:21, Daniel Carr wrote:
> Hi all.
>
> I'm using couchDB 1.2.1
>
> I have a replication that I start by posting to _replicate. When I poll
> _active_tasks the progress gets stuck on 98, and will not go up. This is
> despite all the documents being replicated.
>
> It appears that the source_seq is higher than it should be. As when I poll
> the remotehost _changes feed, it gives a lower last_seq than the source_seq
> in localhost's _active_tasks.
>
> Any ideas what's going on here?
> My code relies on the progress getting to 100, should I not do this?
>
> Cheers,
> Daniel
>
>
> I've looked the entry in the log showing the start of the replication, but
> can't find it. If it would be helpful, please tell me what to search for.
>
>
> curl http://localhost:5984/_active_tasks:
>
> [{
> checkpointed_source_seq: 251
> continuous: true
> doc_id: null
> doc_write_failures: 0
> docs_read: 150
> docs_written: 150
> missing_revisions_found: 150
> pid: "<0.502.0>"
> progress: 98
> replication_id: "afeb3a2e577aee71c767ed96090505c6+continuous"
> revisions_checked: 150
> source: "http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/"
> source_seq: 255
> started_on: 1378331698
> target: "aml"
> type: "replication"
> updated_on: 1378331916
> }]
>
> curl
> http://remotehost:5984/d59423696-9e74-e111-b767-00215aac8bce-aml/_changes :
>
> {"results":[
> ...
> {"seq":251,"id":"fc47efd9448a47b990b01a412f7da8ce","changes":[{"rev":"1-65d41b82c27c6df7891a6bd875b18c1c"}]}
> ],
> "last_seq":251}
>
>