You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Ning Tan (JIRA)" <ji...@apache.org> on 2009/10/01 18:35:23 UTC
[jira] Created: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
------------------------------------------------------------------------------------------------
Key: COUCHDB-516
URL: https://issues.apache.org/jira/browse/COUCHDB-516
Project: CouchDB
Issue Type: Bug
Components: Database Core
Affects Versions: 0.10
Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
Reporter: Ning Tan
When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Resolved: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by Ning Tan <ni...@gmail.com>.
On Thu, Oct 1, 2009 at 8:33 PM, Adam Kocoloski (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Adam Kocoloski resolved COUCHDB-516.
> ------------------------------------
>
> Resolution: Fixed
> Fix Version/s: 0.10
> Assignee: Adam Kocoloski
>
> Hi Ning, Matt, this should be fixed now (in trunk and 0.10.x). Can you confirm?
It looks promising. I have not been able to reproduce the problem.
[jira] Updated: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by "Ning Tan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Tan updated COUCHDB-516:
-----------------------------
Attachment: replication.txt
Replication records and corresponding server logs.
> Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
> ------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-516
> URL: https://issues.apache.org/jira/browse/COUCHDB-516
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.10
> Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
> Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
> Reporter: Ning Tan
> Attachments: replication.txt
>
>
> When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
> The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
> It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
> Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski resolved COUCHDB-516.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.10
Assignee: Adam Kocoloski
Hi Ning, Matt, this should be fixed now (in trunk and 0.10.x). Can you confirm?
> Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
> ------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-516
> URL: https://issues.apache.org/jira/browse/COUCHDB-516
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.10
> Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
> Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
> Reporter: Ning Tan
> Assignee: Adam Kocoloski
> Fix For: 0.10
>
> Attachments: replication.txt
>
>
> When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
> The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
> It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
> Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by "Matt Goodall (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761268#action_12761268 ]
Matt Goodall commented on COUCHDB-516:
--------------------------------------
Also seen replicating between two Ubuntu machines (one Jaunty, one Karmic) both running exactly the same version of CouchDB (git 9dfbf2ca3a9028262dff8e21637e41108701e9c3 from the 0.10.x branch, 820498 from svn 0.10.x branch). Both machines on same network with no proxy, firewall, etc in between. Replication was not continuous.
In my case the replication process did not end and I had to restart the CouchDB server to continue.
> Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
> ------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-516
> URL: https://issues.apache.org/jira/browse/COUCHDB-516
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.10
> Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
> Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
> Reporter: Ning Tan
> Attachments: replication.txt
>
>
> When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
> The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
> It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
> Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by "Matt Goodall (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12761514#action_12761514 ]
Matt Goodall commented on COUCHDB-516:
--------------------------------------
For reference, the problem did not occur for me when using continuous replication.
I've upgraded both machines to svn r 820852 and can confirm replication now runs to completion for non-continuous and continuous replication.
Thanks Adam.
> Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
> ------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-516
> URL: https://issues.apache.org/jira/browse/COUCHDB-516
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.10
> Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
> Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
> Reporter: Ning Tan
> Assignee: Adam Kocoloski
> Fix For: 0.10
>
> Attachments: replication.txt
>
>
> When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
> The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
> It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
> Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-516) Replication: _replicate does not
finish replication in one pass and has to be invoked repeatedly
Posted by "Adam Kocoloski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Kocoloski closed COUCHDB-516.
----------------------------------
Ning Tan replied on dev@ that the bug is resolved, so we can close this out.
> Replication: _replicate does not finish replication in one pass and has to be invoked repeatedly
> ------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-516
> URL: https://issues.apache.org/jira/browse/COUCHDB-516
> Project: CouchDB
> Issue Type: Bug
> Components: Database Core
> Affects Versions: 0.10
> Environment: Mac OS X 10.5 w/ CouchDB 0.10.x multiple revisions (latest is 820436)
> Ubuntu Desktop 9.04 w/ CouchDB 0.10.x multiple revisions (latest is 818247)
> Reporter: Ning Tan
> Assignee: Adam Kocoloski
> Fix For: 0.10
>
> Attachments: replication.txt
>
>
> When we replicate between a remote database and a local one (pulling from remote into local), we are observing partial replications, meaning that we have to issue repeated _replicate calls for the replication to complete. For a database with 17,000 documents, for example, it could take up to 7 calls for the entire database to replicate into an empty one. Each time, the number of documents replicated over seemed random.
> The use case is very simple--replicating a database into an empty one with no concurrent writes, no additional load or i/o, etc. The databases involved are a mixture of the 10.0.x code base natively built on Ubuntu and Mac.
> It seems to me that every (not all) partial replication process is associated with a corresponding entry in the log that says "recording a checkpoint at source update_seq .....". (i.e. you can match the recorded_seq number in the replication response with the checkpoint update_seq numbers in the log).
> Futon vs. curl doesn't make a difference.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.