You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Nicolas Maupu (JIRA)" <ji...@apache.org> on 2011/02/01 18:16:29 UTC
[jira] Created: (COUCHDB-1052) CouchDB crashes by Erlang Heartbeat
Runtime System
CouchDB crashes by Erlang Heartbeat Runtime System
--------------------------------------------------
Key: COUCHDB-1052
URL: https://issues.apache.org/jira/browse/COUCHDB-1052
Project: CouchDB
Issue Type: Improvement
Components: Infrastructure
Affects Versions: 1.0.1, 0.10.1
Environment: Solaris 10
Reporter: Nicolas Maupu
Hello,
We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Noah Slater (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989306#comment-12989306 ]
Noah Slater commented on COUCHDB-1052:
--------------------------------------
There is no such thing as a graceful restart for CouchDB. CouchDB has a crash-only design. So yes, the HEARTBEAT_COMMAND is correct in crashing CouchDB when it thinks something is up. The real question here is: why is CouchDB non-responsive in the first place? Something is clearly up with that.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Nicolas Maupu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997483#comment-12997483 ]
Nicolas Maupu commented on COUCHDB-1052:
----------------------------------------
Hi,
I successfully test RESPAWN_TIMEOUT to something non-zero. It works as expected (when calling startup script with -R).
To avoid editing couchdb scripts to change this value, it should be a good idea to declare bash variable like this :
RESPAWN_TIMEOUT=${RESPAWN_TIMEOUT:-0}
This way, we can pass RESPAWN_TIMEOUT as an env var when calling couchdb script. Starting couchdb could look like this :
RESPAWN_TIMEOUT=1 /opt/couchdb/bin/couchdb -b -R
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990300#comment-12990300 ]
Paul Joseph Davis commented on COUCHDB-1052:
--------------------------------------------
http://www.youtube.com/watch?v=JmyXTOHC3w8&t=8m45s
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990208#comment-12990208 ]
Paul Joseph Davis commented on COUCHDB-1052:
--------------------------------------------
Noper. Its just a matter of making sure that the Erlang VM isn't trapping the signal we use. IIRC, SIGINT and SIGKILL are untrappable so either of those should be fine.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Noah Slater (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990091#comment-12990091 ]
Noah Slater commented on COUCHDB-1052:
--------------------------------------
SIGKILL would be good as long as there are no adverse side-effects. Does CouchDB do ANY sort of additional bookkeeping with a SIGHUP etc that we might want to take advantage of?
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990294#comment-12990294 ]
Randall Leeds commented on COUCHDB-1052:
----------------------------------------
SIGKILL and SIGSTOP are untrappable. According to my `man kill`, SIGSTOP stops the process, but doesn't tell it to exit. Is this what C-z does? Anyway. I vote for KILL KILL KILL!!!!
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989761#comment-12989761 ]
Randall Leeds commented on COUCHDB-1052:
----------------------------------------
I meant RESPAWN_TIMEOUT
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (COUCHDB-1052) CouchDB crashes by Erlang Heartbeat
Runtime System
Posted by "Nicolas Maupu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Maupu updated COUCHDB-1052:
-----------------------------------
Attachment: couchdb-bin-1.0.1.diff
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989760#comment-12989760 ]
Randall Leeds commented on COUCHDB-1052:
----------------------------------------
Nicolas: I believe what you want is to set RESTART_TIMEOUT to something non-zero. This change will cause CouchDB to restart automatically after a crash. You get the restart behavior you want and also protection against other crashes. I think this should work for you.
The point you raise about signal 1 vs 15 is a good one though. Should we not just use 9? Crash Only means Kill Aggressively, no? ;)
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Alexander (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456479#comment-13456479 ]
Alexander commented on COUCHDB-1052:
------------------------------------
Please, reopen this bug - we need a quick solution for CouchDB 1.0.3.
All our webservers are down!
RESPAWN_TIMEOUT=1 doesn't work and we can't to apply an attached patch cause couchdb 1.0.3 has another code lines.
Waiting for your response.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COUCHDB-1052) CouchDB crashes by Erlang Heartbeat
Runtime System
Posted by "Jan Lehnardt (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lehnardt closed COUCHDB-1052.
---------------------------------
Resolution: Not A Problem
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Alexander (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456512#comment-13456512 ]
Alexander commented on COUCHDB-1052:
------------------------------------
Additional note:
We just moved to CouchDB 1.2.0 with spidermonkey 1.7.0 and all works (with spidermonkey 1.8.5 there is a problem).
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990209#comment-12990209 ]
Paul Joseph Davis commented on COUCHDB-1052:
--------------------------------------------
Wait, no, not SIGINT, but I seem to recall two that the kernel doesn't allow a process to override.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989299#comment-12989299 ]
Paul Joseph Davis commented on COUCHDB-1052:
--------------------------------------------
How do you mean graceful restart of CouchDB? Did you find a ticket for heart that had to do with errors with system clocks? I can imagine there'd be weirdness with clocks that change. I know there used to be a thing when laptops woke up after sleeping, but AFAIK we fixed that.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Nicolas Maupu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989570#comment-12989570 ]
Nicolas Maupu commented on COUCHDB-1052:
----------------------------------------
What I mean by graceful restart is just to be sure CouchDB restart after a erlang "heart attack" instead of stop
I suspect system clock to cause this erlang event because I don't have any NTP server (and can't use it at the moment).
I added patch I use in attached files.
I changed HEART_COMMAND and start / stop functions. I also changed signal sent to PID to stop (1 to 15) because on my system, this signal did not stop process as expected.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998058#comment-12998058 ]
Randall Leeds commented on COUCHDB-1052:
----------------------------------------
bin/couchdb includes the option "-r" for passing a respawn timeout on the command line.
The init script uses an environment variable called COUCHDB_RESPAWN_TIMEOUT which is passed to /usr/bin/couchdb with this option.
If you use the init script, you'll notice that it will source a file (in your case, given your installation path, maybe /opt/couchdb/etc/default/couchdb) to get the environment.
COUCHDB_RESPAWN_TIMEOUT is set in there if you'd like to use the init script.
Otherwise, calling couchdb with "-r 1" should do what you want.
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Attachments: couchdb-bin-1.0.1.diff
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (COUCHDB-1052) CouchDB crashes by Erlang
Heartbeat Runtime System
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989300#comment-12989300 ]
Paul Joseph Davis commented on COUCHDB-1052:
--------------------------------------------
Also, can you paste a diff of the patch you're using?
> CouchDB crashes by Erlang Heartbeat Runtime System
> --------------------------------------------------
>
> Key: COUCHDB-1052
> URL: https://issues.apache.org/jira/browse/COUCHDB-1052
> Project: CouchDB
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: 0.10.1, 1.0.1
> Environment: Solaris 10
> Reporter: Nicolas Maupu
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Hello,
> We have several couchdb in our production environment (starting from 0.10.1 and upgraded to 1.0.1). Some instances began to crash several times a day.
> After research (I am not aware of all erlang tips and tricks), I found that the problem was caused by erlang heartbeat runtime system (http://www.erlang.org/doc/man/heart.html).
> For some reasons (it seems to be a problem with system clock), this heartbeat process sends a signal to main erlang process to trigger HEARTBEAT_COMMAND.
> The command configured by couchdb startup script is to call 'bin/couchdb -k' which kill server ...
> I don't know if this is wanted or not but is Erlang heartbeat system provided to restart a service if this one crash ?
> I patched mine to restart process gracefully but I would like to know if this is a feature or a bug : next time I upgrade, I will have to patch again.
> My patch is a little bit specific to my architecture so I did not provide it. Ask me if you are interested.
> nm.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira