You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Marco Monteiro (JIRA)" <ji...@apache.org> on 2010/09/25 12:28:32 UTC
[jira] Created: (COUCHDB-892) Method for restarting filtered
replication
Method for restarting filtered replication
------------------------------------------
Key: COUCHDB-892
URL: https://issues.apache.org/jira/browse/COUCHDB-892
Project: CouchDB
Issue Type: New Feature
Components: Replication
Reporter: Marco Monteiro
Priority: Minor
I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
were created before the last replication are not replicated. So, I have to remove the database and recreate it.
I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914810#action_12914810 ]
Randall Leeds commented on COUCHDB-892:
---------------------------------------
I second what Bob said ("I'm surprised..."). I shouldn't be surprised, because I've seen this code a lot, but oh well.
But I also think a start_seq might turn out to be useful in other situations in the future:
a) an administrator changes the hostname of a box and wants to re-start replication without starting from the beginning again.
b) fast-forward replication from a source when you trust a third party from which you replicated the source.
c) scp a .couch file to create a redundant copy and jump-start continuous replication.
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914829#action_12914829 ]
Filipe Manana commented on COUCHDB-892:
---------------------------------------
Randall,
good points.
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914804#action_12914804 ]
Robert Newson commented on COUCHDB-892:
---------------------------------------
I'm surprised we don't already include "the entire string of the filter function ... to calculate the replication ID", which sounds like the better fix and aligns with the way views are named.
The problem here is that the target database's checkpoint would depend on state of the source database? Is the concern that this has security implications? The replicator, if started at the target end, would be required to read the filter of the ddoc, replicate changes, and abort if the ddoc changes during the replication.
adding start_seq is simpler but including the function in the MD5 calculation sounds more correct.
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Joseph Davis updated COUCHDB-892:
--------------------------------------
Skill Level: Committers Level (Medium to Hard)
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915830#action_12915830 ]
Filipe Manana commented on COUCHDB-892:
---------------------------------------
A fix is committed to trunk, revision 1002259.
As for the start_seq parameter, it will come later.
Still would like to have opinion from others regarding it, specially from Adam.
Marco, can you test it?
regards
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Marco Monteiro (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919810#action_12919810 ]
Marco Monteiro commented on COUCHDB-892:
----------------------------------------
The query_params argument can be used to parameterize the filter function.
Shouldn't the query_params string also be included in the function to calculate the replication ID?
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914801#action_12914801 ]
Filipe Manana commented on COUCHDB-892:
---------------------------------------
Adding the start_seq field to a replication object is the simplest way of doing it.
Otherwise the replicator must use the entire string of the filter function (which resides in a design doc of the source DB) as part of the input to the MD5 function used to calculate the replication ID.
Please, give your opinions.
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Filipe Manana (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filipe Manana reassigned COUCHDB-892:
-------------------------------------
Assignee: Filipe Manana
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-892) Method for restarting filtered
replication
Posted by "Marco Monteiro (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919815#action_12919815 ]
Marco Monteiro commented on COUCHDB-892:
----------------------------------------
Please, ignore my last comment.
The query_params string is already included in the calculation.
> Method for restarting filtered replication
> ------------------------------------------
>
> Key: COUCHDB-892
> URL: https://issues.apache.org/jira/browse/COUCHDB-892
> Project: CouchDB
> Issue Type: New Feature
> Components: Replication
> Reporter: Marco Monteiro
> Assignee: Filipe Manana
> Priority: Minor
>
> I'm using filtered replication to maitain a database that has a subset of the documents of some other database.
> The problem I have is when I change the filter. If the filter allows for more documents to be replicated, those that
> were created before the last replication are not replicated. So, I have to remove the database and recreate it.
> I see two solutions that can be implemented to help in this situation. The first one is for the filtered replication
> to restart whenever the filter changes (maybe also adding an argument to replicate to tell that" we want this to happen).
> Another solution is to add an argument to replicate ("start_seq", as sugested by fdmanana on IRC) that would be the
> source database ""local_seq that we want the replication to start from. This an be set to 1 to restart from first change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.