You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Jason Woods <de...@jasonwoods.me.uk> on 2014/08/06 13:48:31 UTC
Crashing couchdb
Hi all,
Hopefully someone can help shed some light on this. The logs aren't the easiest thing to understand :(
CouchDB is 1.6.0. Thousands of document, all are several MB in size.
When we put a little bit of a read strain on it, it appears that memory consumption climbs to ridiculous proportions, around 5-6GB.
Then it crashes completely. A few errors appear and then all processes end.
Before we upgraded to 1.6.0 we were using 1.3.1, and under the same conditions, it would continuously crash and restart processes, and eventually OOM-killer kicks in and destroys everything nice in the world. Since upgrading to 1.6.0 this morning it appears to just completely crash instead.
When I say bit of a read strain... to me it looks honestly like a maximum of 1 or 2 reads a second maximum. Though the transfer time will potentially be long due to the several MB size.
Here is what appears in the log a few seconds before the crash:
[Wed, 06 Aug 2014 11:31:03 GMT] [error] [<0.531.0>] {error_report,<0.30.0>,
{<0.531.0>,crash_report,
[[{initial_call,
{mochiweb_acceptor,init,
['Argument__1','Argument__2','Argument__3']}},
{pid,<0.531.0>},
{registered_name,[]},
{error_info,
{exit,
{timeout,
{gen_server,call,[couch_httpd_vhost,get_state]}},
[{gen_server,call,2},
{couch_httpd_vhost,dispatch_host,1},
{couch_httpd,handle_request,5},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_httpd,couch_secondary_services,
couch_server_sup,<0.31.0>]},
{messages,
[{#Ref<0.0.0.3824>,
{vhosts_state,[],
["_utils","_uuids","_session","_oauth","_users"],
#Fun<couch_httpd.8.25236666>}}]},
{links,[<0.126.0>,#Port<0.3479>]},
{dictionary,[{couch_rewrite_count,0}]},
{trap_exit,false},
{status,running},
{heap_size,2584},
{stack_size,24},
{reductions,941}],
[]]}}
[Wed, 06 Aug 2014 11:31:31 GMT] [error] [<0.530.0>] {error_report,<0.30.0>,
{<0.530.0>,crash_report,
[[{initial_call,
{mochiweb_acceptor,init,
['Argument__1','Argument__2','Argument__3']}},
{pid,<0.530.0>},
{registered_name,[]},
{error_info,
{exit,
{timeout,
{gen_server,call,[couch_httpd_vhost,get_state]}},
[{gen_server,call,2},
{couch_httpd_vhost,dispatch_host,1},
{couch_httpd,handle_request,5},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_httpd,couch_secondary_services,
couch_server_sup,<0.31.0>]},
{messages,
[{#Ref<0.0.0.3825>,
{vhosts_state,[],
["_utils","_uuids","_session","_oauth","_users"],
#Fun<couch_httpd.8.25236666>}}]},
{links,[<0.126.0>,#Port<0.3480>]},
{dictionary,[{couch_rewrite_count,0}]},
{trap_exit,false},
{status,running},
{heap_size,2584},
{stack_size,24},
{reductions,941}],
[]]}}
Any advice on diagnosing is gratefully appreciated.
Regards,
Jason
Re: Crashing couchdb
Posted by Jason Woods <de...@jasonwoods.me.uk>.
Hi
On 6 Aug 2014, at 12.59, Robert Newson <rn...@apache.org> wrote:
> Default timeout in vhost module is a bug. 5s not long enough for that.
>
> Sent from my iPhone
Should the entire process crash though? And is the huge memory usage normal?
I will try adjusting the timeout though and see what happens.
Thanks!
Jason
Re: Crashing couchdb
Posted by Robert Newson <rn...@apache.org>.
Default timeout in vhost module is a bug. 5s not long enough for that.
Sent from my iPhone
> On 6 Aug 2014, at 12:48, Jason Woods <de...@jasonwoods.me.uk> wrote:
>
> Hi all,
>
> Hopefully someone can help shed some light on this. The logs aren't the easiest thing to understand :(
> CouchDB is 1.6.0. Thousands of document, all are several MB in size.
>
> When we put a little bit of a read strain on it, it appears that memory consumption climbs to ridiculous proportions, around 5-6GB.
> Then it crashes completely. A few errors appear and then all processes end.
> Before we upgraded to 1.6.0 we were using 1.3.1, and under the same conditions, it would continuously crash and restart processes, and eventually OOM-killer kicks in and destroys everything nice in the world. Since upgrading to 1.6.0 this morning it appears to just completely crash instead.
>
> When I say bit of a read strain... to me it looks honestly like a maximum of 1 or 2 reads a second maximum. Though the transfer time will potentially be long due to the several MB size.
>
> Here is what appears in the log a few seconds before the crash:
>
> [Wed, 06 Aug 2014 11:31:03 GMT] [error] [<0.531.0>] {error_report,<0.30.0>,
> {<0.531.0>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2','Argument__3']}},
> {pid,<0.531.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {timeout,
> {gen_server,call,[couch_httpd_vhost,get_state]}},
> [{gen_server,call,2},
> {couch_httpd_vhost,dispatch_host,1},
> {couch_httpd,handle_request,5},
> {mochiweb_http,headers,5},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.31.0>]},
> {messages,
> [{#Ref<0.0.0.3824>,
> {vhosts_state,[],
> ["_utils","_uuids","_session","_oauth","_users"],
> #Fun<couch_httpd.8.25236666>}}]},
> {links,[<0.126.0>,#Port<0.3479>]},
> {dictionary,[{couch_rewrite_count,0}]},
> {trap_exit,false},
> {status,running},
> {heap_size,2584},
> {stack_size,24},
> {reductions,941}],
> []]}}
> [Wed, 06 Aug 2014 11:31:31 GMT] [error] [<0.530.0>] {error_report,<0.30.0>,
> {<0.530.0>,crash_report,
> [[{initial_call,
> {mochiweb_acceptor,init,
> ['Argument__1','Argument__2','Argument__3']}},
> {pid,<0.530.0>},
> {registered_name,[]},
> {error_info,
> {exit,
> {timeout,
> {gen_server,call,[couch_httpd_vhost,get_state]}},
> [{gen_server,call,2},
> {couch_httpd_vhost,dispatch_host,1},
> {couch_httpd,handle_request,5},
> {mochiweb_http,headers,5},
> {proc_lib,init_p_do_apply,3}]}},
> {ancestors,
> [couch_httpd,couch_secondary_services,
> couch_server_sup,<0.31.0>]},
> {messages,
> [{#Ref<0.0.0.3825>,
> {vhosts_state,[],
> ["_utils","_uuids","_session","_oauth","_users"],
> #Fun<couch_httpd.8.25236666>}}]},
> {links,[<0.126.0>,#Port<0.3480>]},
> {dictionary,[{couch_rewrite_count,0}]},
> {trap_exit,false},
> {status,running},
> {heap_size,2584},
> {stack_size,24},
> {reductions,941}],
> []]}}
>
> Any advice on diagnosing is gratefully appreciated.
>
> Regards,
>
> Jason