You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Germain Maurice <ge...@linkfluence.net> on 2010/03/25 01:20:51 UTC
Cancelling a replication on 0.11.0
Hi, still me with my 0.11.0 couchdb :)
I launched continuous replication between two hosts (more than 8
millions of documents), it takes long time and i'm ok with that.
My problem is that i launched another one shot replication between the
same databases as the previous replication.
So, i looked after a command to cancel the "one shot" replication (via
Jira COUCHDB-664), but it does not work, my command is :
curl -X POST http://hostA:5984/_replicate -d
'{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
Response is : {"error":"not_found"}
How to cancel the task ? I have PID number from the "Status" page on
Futon, how to use it ?
Regards,
Germain
Re: Cancelling a replication on 0.11.0
Posted by Germain Maurice <ge...@linkfluence.net>.
Really thank you Randall, you're right it will be useful to have these
informations on the Wiki,
especially because we don't understand Erlang :)
I don't know if the way of replication is definitely defined, but i will
be very useful to know
that... It will permit us to design more sharper our couchdb infrastructure.
Randall Leeds a écrit :
> W means "Writer" and MR means "Missing Revs".
> These reference two pieces of the replication task and each one checks
> in with the main replication task to mark checkpoints.
>
> MR Processed source update #XXX means that up to sequence number XXX
> couch has checked and knows whether those revisions are missing and
> need to be fetched.
>
> W Processed source update #XXX means that up to sequence number XXX
> couch has found and retrieved all missing revisions and written them
> to the target database.
>
> Both situations represent some sort of "waiting" state in that they
> won't update until the next checkpoint.
> If a replication stays in MR it means that either there were no
> missing revisions in the last batch (so nothing was fetched and sent
> to the writer) or the writer has not finished writing the batch yet
> and it will soon be updated to show W.
>
> I understand this is totally impossible to know right now since I
> don't think it's documented anywhere. I found this out by reading the
> code. Maybe I'll take a moment this weekend to put it in the wiki.
>
> -Randall
>
> On Thu, Mar 25, 2010 at 03:10, Germain Maurice
> <ge...@linkfluence.net> wrote:
>
>> I think i'm a bit tired because 'W' could probably mean "Waiting".
>> I just saw that the database and its replicate were successfully replicated
>> (same number of documents and same update seq), so, my continuous
>> replication would naturally wait for changes.
>>
>> A compaction on a database which is replicated in "push" mode will work fine
>> ?
>> i'm a bit nervous because the file database is huge and it takes many hours
>> to recover it.
>>
>> I hope you're still patient with me :)
>>
>> Germain Maurice a écrit :
>>
>>> Thank you for your replies.
>>>
>>> However, I would like to have some explainations on replication, here is
>>> my question :
>>>
>>> On hostA, i have two replication processes, the first one is continuous,
>>> the second is one shot (a mistake).
>>>
>>> Replication ac88d7: blog_content ->
>>> http://hostB:5984/database/<0.4334.0> W Processed source update
>>> #9219757
>>> Replication b590d2: blog_content ->
>>> http://hostB:5984/blog_content/<0.27442.2> MR Processed source update
>>> #4426465
>>>
>>> Can you explain what do 'W' and 'MR' before "Processed source update" mean
>>> ?
>>> I think this answer could explain why second replication is faster than
>>> the first .. :)
>>>
>>> Thanks
>>>
>>> Randall Leeds a écrit :
>>>
>>>> On Wed, Mar 24, 2010 at 22:01, Jan Lehnardt <ja...@apache.org> wrote:
>>>>
>>>>
>>>>> On 24 Mar 2010, at 17:20, Germain Maurice wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi, still me with my 0.11.0 couchdb :)
>>>>>>
>>>>>> I launched continuous replication between two hosts (more than 8
>>>>>> millions of documents), it takes long time and i'm ok with that.
>>>>>> My problem is that i launched another one shot replication between the
>>>>>> same databases as the previous replication.
>>>>>>
>>>>>> So, i looked after a command to cancel the "one shot" replication (via
>>>>>> Jira COUCHDB-664), but it does not work, my command is :
>>>>>>
>>>>>> curl -X POST http://hostA:5984/_replicate -d
>>>>>> '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>>>>>>
>>>>>> Response is : {"error":"not_found"}
>>>>>>
>>>>>>
>>>>> Sorry to point out the obvious, but can you double check, that the -d
>>>>> {}-object values are identical to the ones in the replication you want to
>>>>> cancel?
>>>>>
>>>>>
>>>>>
>>>>>> How to cancel the task ? I have PID number from the "Status" page on
>>>>>> Futon, how to use it ?
>>>>>>
>>>>>>
>>>>> killing with the pid is a bit tricky and requires Erlang-fu. I've spent
>>>>> 15 minutes earlier today to kill a long running view process and got stuck
>>>>> and then run out of time. That said, with distributed Erlang and the rpc
>>>>> module, this should be doable.
>>>>>
>>>>>
>>>> First you have to start erlang in distributed mode, which couch does
>>>> not do by default. ERL_FLAGS="-name couch -setcookie secret" should do
>>>> the trick. Then you can connect to the node with "erl -name repkiller
>>>> -setcookie secret -remsh couch@fqdn.of.couch.node". Starting up appmon
>>>> with appmon:start() should give you a nice graphical way to browse the
>>>> supervisor tree and make it easy to identify the replication.
>>>>
>>>> But if you didn't already modify couch to start with a distributed
>>>> erlang name and cookie you're stuck and you'll have to restart couchdb
>>>> anyway. Could be useful in the future if you want to kill anything.
>>>> Murderer.
>>>>
>>>>
>>>
>> --
>> Germain Maurice
>> Administrateur Système/Réseau
>> Tel : +33.(0)1.42.43.54.33
>>
>> http://www.linkfluence.net
>> **linkfluence news & events**
>> 2009 excellence award nominee from ESOMAR
>> 2009 marketing research silver award from semo & marketing magazine (France)
>> 2009 european excellence award recipient (PR evaluation, wahlradar.de, joint
>> project with Publicis Consultants)
>>
>>
>>
--
Germain Maurice
Administrateur Système/Réseau
Tel : +33.(0)1.42.43.54.33
http://www.linkfluence.net
**linkfluence news & events**
2009 excellence award nominee from ESOMAR
2009 marketing research silver award from semo & marketing magazine (France)
2009 european excellence award recipient (PR evaluation, wahlradar.de, joint project with Publicis Consultants)
Re: Cancelling a replication on 0.11.0
Posted by Randall Leeds <ra...@gmail.com>.
W means "Writer" and MR means "Missing Revs".
These reference two pieces of the replication task and each one checks
in with the main replication task to mark checkpoints.
MR Processed source update #XXX means that up to sequence number XXX
couch has checked and knows whether those revisions are missing and
need to be fetched.
W Processed source update #XXX means that up to sequence number XXX
couch has found and retrieved all missing revisions and written them
to the target database.
Both situations represent some sort of "waiting" state in that they
won't update until the next checkpoint.
If a replication stays in MR it means that either there were no
missing revisions in the last batch (so nothing was fetched and sent
to the writer) or the writer has not finished writing the batch yet
and it will soon be updated to show W.
I understand this is totally impossible to know right now since I
don't think it's documented anywhere. I found this out by reading the
code. Maybe I'll take a moment this weekend to put it in the wiki.
-Randall
On Thu, Mar 25, 2010 at 03:10, Germain Maurice
<ge...@linkfluence.net> wrote:
> I think i'm a bit tired because 'W' could probably mean "Waiting".
> I just saw that the database and its replicate were successfully replicated
> (same number of documents and same update seq), so, my continuous
> replication would naturally wait for changes.
>
> A compaction on a database which is replicated in "push" mode will work fine
> ?
> i'm a bit nervous because the file database is huge and it takes many hours
> to recover it.
>
> I hope you're still patient with me :)
>
> Germain Maurice a écrit :
>>
>> Thank you for your replies.
>>
>> However, I would like to have some explainations on replication, here is
>> my question :
>>
>> On hostA, i have two replication processes, the first one is continuous,
>> the second is one shot (a mistake).
>>
>> Replication ac88d7: blog_content ->
>> http://hostB:5984/database/<0.4334.0> W Processed source update
>> #9219757
>> Replication b590d2: blog_content ->
>> http://hostB:5984/blog_content/<0.27442.2> MR Processed source update
>> #4426465
>>
>> Can you explain what do 'W' and 'MR' before "Processed source update" mean
>> ?
>> I think this answer could explain why second replication is faster than
>> the first .. :)
>>
>> Thanks
>>
>> Randall Leeds a écrit :
>>>
>>> On Wed, Mar 24, 2010 at 22:01, Jan Lehnardt <ja...@apache.org> wrote:
>>>
>>>>
>>>> On 24 Mar 2010, at 17:20, Germain Maurice wrote:
>>>>
>>>>
>>>>>
>>>>> Hi, still me with my 0.11.0 couchdb :)
>>>>>
>>>>> I launched continuous replication between two hosts (more than 8
>>>>> millions of documents), it takes long time and i'm ok with that.
>>>>> My problem is that i launched another one shot replication between the
>>>>> same databases as the previous replication.
>>>>>
>>>>> So, i looked after a command to cancel the "one shot" replication (via
>>>>> Jira COUCHDB-664), but it does not work, my command is :
>>>>>
>>>>> curl -X POST http://hostA:5984/_replicate -d
>>>>> '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>>>>>
>>>>> Response is : {"error":"not_found"}
>>>>>
>>>>
>>>> Sorry to point out the obvious, but can you double check, that the -d
>>>> {}-object values are identical to the ones in the replication you want to
>>>> cancel?
>>>>
>>>>
>>>>>
>>>>> How to cancel the task ? I have PID number from the "Status" page on
>>>>> Futon, how to use it ?
>>>>>
>>>>
>>>> killing with the pid is a bit tricky and requires Erlang-fu. I've spent
>>>> 15 minutes earlier today to kill a long running view process and got stuck
>>>> and then run out of time. That said, with distributed Erlang and the rpc
>>>> module, this should be doable.
>>>>
>>>
>>> First you have to start erlang in distributed mode, which couch does
>>> not do by default. ERL_FLAGS="-name couch -setcookie secret" should do
>>> the trick. Then you can connect to the node with "erl -name repkiller
>>> -setcookie secret -remsh couch@fqdn.of.couch.node". Starting up appmon
>>> with appmon:start() should give you a nice graphical way to browse the
>>> supervisor tree and make it easy to identify the replication.
>>>
>>> But if you didn't already modify couch to start with a distributed
>>> erlang name and cookie you're stuck and you'll have to restart couchdb
>>> anyway. Could be useful in the future if you want to kill anything.
>>> Murderer.
>>>
>>
>>
>
>
> --
> Germain Maurice
> Administrateur Système/Réseau
> Tel : +33.(0)1.42.43.54.33
>
> http://www.linkfluence.net
> **linkfluence news & events**
> 2009 excellence award nominee from ESOMAR
> 2009 marketing research silver award from semo & marketing magazine (France)
> 2009 european excellence award recipient (PR evaluation, wahlradar.de, joint
> project with Publicis Consultants)
>
>
Re: Cancelling a replication on 0.11.0
Posted by Germain Maurice <ge...@linkfluence.net>.
I think i'm a bit tired because 'W' could probably mean "Waiting".
I just saw that the database and its replicate were successfully
replicated (same number of documents and same update seq), so, my
continuous replication would naturally wait for changes.
A compaction on a database which is replicated in "push" mode will work
fine ?
i'm a bit nervous because the file database is huge and it takes many
hours to recover it.
I hope you're still patient with me :)
Germain Maurice a écrit :
> Thank you for your replies.
>
> However, I would like to have some explainations on replication, here
> is my question :
>
> On hostA, i have two replication processes, the first one is
> continuous, the second is one shot (a mistake).
>
> Replication ac88d7: blog_content ->
> http://hostB:5984/database/<0.4334.0> W Processed source
> update #9219757
> Replication b590d2: blog_content ->
> http://hostB:5984/blog_content/<0.27442.2> MR Processed source
> update #4426465
>
> Can you explain what do 'W' and 'MR' before "Processed source update"
> mean ?
> I think this answer could explain why second replication is faster
> than the first .. :)
>
> Thanks
>
> Randall Leeds a écrit :
>> On Wed, Mar 24, 2010 at 22:01, Jan Lehnardt <ja...@apache.org> wrote:
>>
>>> On 24 Mar 2010, at 17:20, Germain Maurice wrote:
>>>
>>>
>>>> Hi, still me with my 0.11.0 couchdb :)
>>>>
>>>> I launched continuous replication between two hosts (more than 8
>>>> millions of documents), it takes long time and i'm ok with that.
>>>> My problem is that i launched another one shot replication between
>>>> the same databases as the previous replication.
>>>>
>>>> So, i looked after a command to cancel the "one shot" replication
>>>> (via Jira COUCHDB-664), but it does not work, my command is :
>>>>
>>>> curl -X POST http://hostA:5984/_replicate -d
>>>> '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>>>>
>>>>
>>>> Response is : {"error":"not_found"}
>>>>
>>> Sorry to point out the obvious, but can you double check, that the
>>> -d {}-object values are identical to the ones in the replication you
>>> want to cancel?
>>>
>>>
>>>> How to cancel the task ? I have PID number from the "Status" page
>>>> on Futon, how to use it ?
>>>>
>>> killing with the pid is a bit tricky and requires Erlang-fu. I've
>>> spent 15 minutes earlier today to kill a long running view process
>>> and got stuck and then run out of time. That said, with distributed
>>> Erlang and the rpc module, this should be doable.
>>>
>>
>> First you have to start erlang in distributed mode, which couch does
>> not do by default. ERL_FLAGS="-name couch -setcookie secret" should do
>> the trick. Then you can connect to the node with "erl -name repkiller
>> -setcookie secret -remsh couch@fqdn.of.couch.node". Starting up appmon
>> with appmon:start() should give you a nice graphical way to browse the
>> supervisor tree and make it easy to identify the replication.
>>
>> But if you didn't already modify couch to start with a distributed
>> erlang name and cookie you're stuck and you'll have to restart couchdb
>> anyway. Could be useful in the future if you want to kill anything.
>> Murderer.
>>
>
>
--
Germain Maurice
Administrateur Système/Réseau
Tel : +33.(0)1.42.43.54.33
http://www.linkfluence.net
**linkfluence news & events**
2009 excellence award nominee from ESOMAR
2009 marketing research silver award from semo & marketing magazine (France)
2009 european excellence award recipient (PR evaluation, wahlradar.de, joint project with Publicis Consultants)
Re: Cancelling a replication on 0.11.0
Posted by Germain Maurice <ge...@linkfluence.net>.
Thank you for your replies.
However, I would like to have some explainations on replication, here is
my question :
On hostA, i have two replication processes, the first one is continuous,
the second is one shot (a mistake).
Replication ac88d7: blog_content ->
http://hostB:5984/database/<0.4334.0> W Processed source
update #9219757
Replication b590d2: blog_content ->
http://hostB:5984/blog_content/<0.27442.2> MR Processed source update
#4426465
Can you explain what do 'W' and 'MR' before "Processed source update" mean ?
I think this answer could explain why second replication is faster than
the first .. :)
Thanks
Randall Leeds a écrit :
> On Wed, Mar 24, 2010 at 22:01, Jan Lehnardt <ja...@apache.org> wrote:
>
>> On 24 Mar 2010, at 17:20, Germain Maurice wrote:
>>
>>
>>> Hi, still me with my 0.11.0 couchdb :)
>>>
>>> I launched continuous replication between two hosts (more than 8 millions of documents), it takes long time and i'm ok with that.
>>> My problem is that i launched another one shot replication between the same databases as the previous replication.
>>>
>>> So, i looked after a command to cancel the "one shot" replication (via Jira COUCHDB-664), but it does not work, my command is :
>>>
>>> curl -X POST http://hostA:5984/_replicate -d '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>>>
>>> Response is : {"error":"not_found"}
>>>
>> Sorry to point out the obvious, but can you double check, that the -d {}-object values are identical to the ones in the replication you want to cancel?
>>
>>
>>> How to cancel the task ? I have PID number from the "Status" page on Futon, how to use it ?
>>>
>> killing with the pid is a bit tricky and requires Erlang-fu. I've spent 15 minutes earlier today to kill a long running view process and got stuck and then run out of time. That said, with distributed Erlang and the rpc module, this should be doable.
>>
>
> First you have to start erlang in distributed mode, which couch does
> not do by default. ERL_FLAGS="-name couch -setcookie secret" should do
> the trick. Then you can connect to the node with "erl -name repkiller
> -setcookie secret -remsh couch@fqdn.of.couch.node". Starting up appmon
> with appmon:start() should give you a nice graphical way to browse the
> supervisor tree and make it easy to identify the replication.
>
> But if you didn't already modify couch to start with a distributed
> erlang name and cookie you're stuck and you'll have to restart couchdb
> anyway. Could be useful in the future if you want to kill anything.
> Murderer.
>
--
Germain Maurice
Administrateur Système/Réseau
Tel : +33.(0)1.42.43.54.33
http://www.linkfluence.net
**linkfluence news & events**
2009 excellence award nominee from ESOMAR
2009 marketing research silver award from semo & marketing magazine (France)
2009 european excellence award recipient (PR evaluation, wahlradar.de, joint project with Publicis Consultants)
Re: Cancelling a replication on 0.11.0
Posted by Randall Leeds <ra...@gmail.com>.
On Wed, Mar 24, 2010 at 22:01, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 24 Mar 2010, at 17:20, Germain Maurice wrote:
>
>> Hi, still me with my 0.11.0 couchdb :)
>>
>> I launched continuous replication between two hosts (more than 8 millions of documents), it takes long time and i'm ok with that.
>> My problem is that i launched another one shot replication between the same databases as the previous replication.
>>
>> So, i looked after a command to cancel the "one shot" replication (via Jira COUCHDB-664), but it does not work, my command is :
>>
>> curl -X POST http://hostA:5984/_replicate -d '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>>
>> Response is : {"error":"not_found"}
>
> Sorry to point out the obvious, but can you double check, that the -d {}-object values are identical to the ones in the replication you want to cancel?
>
>> How to cancel the task ? I have PID number from the "Status" page on Futon, how to use it ?
>
> killing with the pid is a bit tricky and requires Erlang-fu. I've spent 15 minutes earlier today to kill a long running view process and got stuck and then run out of time. That said, with distributed Erlang and the rpc module, this should be doable.
First you have to start erlang in distributed mode, which couch does
not do by default. ERL_FLAGS="-name couch -setcookie secret" should do
the trick. Then you can connect to the node with "erl -name repkiller
-setcookie secret -remsh couch@fqdn.of.couch.node". Starting up appmon
with appmon:start() should give you a nice graphical way to browse the
supervisor tree and make it easy to identify the replication.
But if you didn't already modify couch to start with a distributed
erlang name and cookie you're stuck and you'll have to restart couchdb
anyway. Could be useful in the future if you want to kill anything.
Murderer.
Re: Cancelling a replication on 0.11.0
Posted by Jan Lehnardt <ja...@apache.org>.
On 24 Mar 2010, at 17:20, Germain Maurice wrote:
> Hi, still me with my 0.11.0 couchdb :)
>
> I launched continuous replication between two hosts (more than 8 millions of documents), it takes long time and i'm ok with that.
> My problem is that i launched another one shot replication between the same databases as the previous replication.
>
> So, i looked after a command to cancel the "one shot" replication (via Jira COUCHDB-664), but it does not work, my command is :
>
> curl -X POST http://hostA:5984/_replicate -d '{"source":"database","target":"http://hostB:5984/database/","cancel":true}'
>
> Response is : {"error":"not_found"}
Sorry to point out the obvious, but can you double check, that the -d {}-object values are identical to the ones in the replication you want to cancel?
> How to cancel the task ? I have PID number from the "Status" page on Futon, how to use it ?
killing with the pid is a bit tricky and requires Erlang-fu. I've spent 15 minutes earlier today to kill a long running view process and got stuck and then run out of time. That said, with distributed Erlang and the rpc module, this should be doable.
Cheers
Jan
--
>
> Regards,
> Germain