couchdb under load

[Norman Barker <no...@gmail.com>]

Hi,

I started testing couchdb when the system was under load from other
processes, basically I wanted to check that couchdb would still work
(this is a real application, not just testing) and couchdb made a few
groans that I have listed below

in

https://svn.apache.org/repos/asf/couchdb/trunk/src/couchdb/couch_os_process.erl

in the init function, BaseProc record doesn't include the timeout
configured in the ini file, this was a simple fix

    Timeout = list_to_integer(couch_config:get(
                            "couchdb", "os_process_timeout", "5000")),
    BaseProc = #os_proc{
        command=Command,
        port=open_port({spawn, Spawnkiller ++ " " ++ Command}, PortOptions),
        writer=fun writejson/2,
        reader=fun readjson/1,
        timeout=Timeout
    },

and then in any gen_server call, the timeout isn't set to infinity, in
particular in couch_query_servers

ret_os_process(Proc) ->
    true = gen_server:call(couch_query_servers, {ret_proc, Proc}, infinity),
    catch unlink(Proc#proc.pid),
    ok.

is there any reason not to make gen_server:call with an infinity timeout?

thanks,

Norman

Re: couchdb under load

[Chris Anderson <jc...@apache.org>]

> is there any reason not to make gen_server:call with an infinity timeout?

I agree I am having a hard time thinking of reasons not to use a long timeout.

I am struggling with similar issues on the iPhone, as performance
there is different from other platforms.

Timeouts provide a check against the system becoming saturated with
unresponsive queries. So it is a prisoners dilemma thing - you never
want your query to timeout. But if no one's ever timed out then a
system under load could become completely unresponsive to all queries.

So it's a trade off I guess. Whenever a timeout bugs me, I tend to
flip it to infinity, but I'm not sure how much things like that are
more specific to resource constrained deployments?

Chris

On Tue, Mar 29, 2011 at 8:29 AM, Norman Barker <no...@gmail.com> wrote:
> Hi,
>
> I started testing couchdb when the system was under load from other
> processes, basically I wanted to check that couchdb would still work
> (this is a real application, not just testing) and couchdb made a few
> groans that I have listed below
>
> in
>
> https://svn.apache.org/repos/asf/couchdb/trunk/src/couchdb/couch_os_process.erl
>
> in the init function, BaseProc record doesn't include the timeout
> configured in the ini file, this was a simple fix
>
>    Timeout = list_to_integer(couch_config:get(
>                            "couchdb", "os_process_timeout", "5000")),
>    BaseProc = #os_proc{
>        command=Command,
>        port=open_port({spawn, Spawnkiller ++ " " ++ Command}, PortOptions),
>        writer=fun writejson/2,
>        reader=fun readjson/1,
>        timeout=Timeout
>    },
>
> and then in any gen_server call, the timeout isn't set to infinity, in
> particular in couch_query_servers
>
> ret_os_process(Proc) ->
>    true = gen_server:call(couch_query_servers, {ret_proc, Proc}, infinity),
>    catch unlink(Proc#proc.pid),
>    ok.
>
>
> thanks,
>
> Norman
>



-- 
Chris Anderson
http://jchrisa.net
http://couchbase.com