You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Sajeesh P N <sa...@tataelxsi.co.in> on 2011/05/03 09:52:36 UTC
Queries on CouchDB tests done using Tsung
Hi,
I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
Thanks,
Sajeesh
Re: Queries on CouchDB tests done using Tsung
Posted by kowsik <ko...@gmail.com>.
If your couch is on the public net, check out http://blitz.io
(couch-powered itself). Also this blog:
http://labs.mudynamics.com/2011/03/15/blitzio-benchmarking-view-performance-in-couchdb/
K.
---
http://blitz.io
http://twitter.com/pcapr
http://labs.mudynamics.com
On Tue, May 3, 2011 at 4:59 AM, Sebastian Cohnen
<se...@googlemail.com> wrote:
> I just found http://wiki.apache.org/couchdb/Performance#System_Resource_Limits - this might be a good starting point.
>
> On 03.05.2011, at 13:53, Sajeesh P N wrote:
>
>> I already tried increasing max_connections to 4096, but still errors are coming after a certain request rate. The max open files in Couchdb server system is just 1024. Should we have to increase this value ?
>>
>> How do we increase the ERL_MAX_PORTS ? If we set this env variable, do we need to set it again for every couchdb restart ? How to see the existing configuration values?
>>
>> Thanks,
>> Sajeesh
>>
>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>> To: <us...@couchdb.apache.org>
>> Sent: Tuesday, May 03, 2011 3:42 PM
>> Subject: Re: Queries on CouchDB tests done using Tsung
>>
>>
>> You could try
>>
>> [httpd]
>> max_connections = 2048
>>
>> Also you could try to increase ERL_MAX_PORTS
>>
>> On 03.05.2011, at 12:03, Sajeesh P N wrote:
>>
>>> Do we need to increase the max connections configuration in Couchdb to increase the no. of requests it can handle. I have 3 Tsung clients, should I have to do any system related configurations in Couchdb like max open files??
>>>
>>> I am facing a problem like, even though I used 3 tsung clients or 1 client, the max no. of requests to Couchdb is around 1000 per sec. I could not increase it further... and getting CouchDB errors like ("mochiweb socket server .... acceptor error... accept failed"). Do you have any idea about this ?
>>>
>>> Thanks,
>>> Sajeesh
>>>
>>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>>> To: <us...@couchdb.apache.org>
>>> Sent: Tuesday, May 03, 2011 3:06 PM
>>> Subject: Re: Queries on CouchDB tests done using Tsung
>>>
>>>
>>> I'm currently doing some performance analysis/engineering for a customer of mine using tsung (not benchmarking couchdb though). To generate load in this magnitude reliably you'll need either more machines generating the load, or doing more requests in each session (even if they are all the same). Using more requests per session vs one request per session has proven to work for me much better - especially when the response time per request is low.
>>>
>>> You should specify you scenarios according to your usage patterns (e.g. read/write ratio, accessing views etc.). Simply hammering on reading docs e.g. is probably not very meaningful for a real world application.
>>>
>>> Another thing you should consider is using a proxy/http accelerator like varnish to serve cacheable resources.
>>>
>>> On 03.05.2011, at 10:31, Sajeesh P N wrote:
>>>
>>>> Thanks. I am reading different documents in Couchdb as my requests. My aim is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB and reported in Tsung. Is it necessary to have more clients in Tsung to generate more requests/sec or have more requests per session for one tsung client.
>>>>
>>>> Thanks in advance,
>>>> Sajeesh
>>>>
>>>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>>>> To: <us...@couchdb.apache.org>
>>>> Sent: Tuesday, May 03, 2011 1:54 PM
>>>> Subject: Re: Queries on CouchDB tests done using Tsung
>>>>
>>>>
>>>> Hey,
>>>>
>>>> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
>>>>>
>>>>> 1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
>>>>
>>>> The default for HTTP load testing in Tsung creates a new connection per request by default, IIRC. But it somehow depends on what you want to test.
>>>>
>>>>> 2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
>>>>
>>>> I'd say that's hard to answer per se. What kind of requests? Reading documents? Creating? Accessing views? ...
>>>>
>>>> If you could elaborate a bit more on what you want to test/benchmark, it might be easier to answer.
>>>>
>>>>
>>>> Best
>>>>
>>>> Sebastian=
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sebastian Cohnen <se...@googlemail.com>.
I just found http://wiki.apache.org/couchdb/Performance#System_Resource_Limits - this might be a good starting point.
On 03.05.2011, at 13:53, Sajeesh P N wrote:
> I already tried increasing max_connections to 4096, but still errors are coming after a certain request rate. The max open files in Couchdb server system is just 1024. Should we have to increase this value ?
>
> How do we increase the ERL_MAX_PORTS ? If we set this env variable, do we need to set it again for every couchdb restart ? How to see the existing configuration values?
>
> Thanks,
> Sajeesh
>
> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
> To: <us...@couchdb.apache.org>
> Sent: Tuesday, May 03, 2011 3:42 PM
> Subject: Re: Queries on CouchDB tests done using Tsung
>
>
> You could try
>
> [httpd]
> max_connections = 2048
>
> Also you could try to increase ERL_MAX_PORTS
>
> On 03.05.2011, at 12:03, Sajeesh P N wrote:
>
>> Do we need to increase the max connections configuration in Couchdb to increase the no. of requests it can handle. I have 3 Tsung clients, should I have to do any system related configurations in Couchdb like max open files??
>>
>> I am facing a problem like, even though I used 3 tsung clients or 1 client, the max no. of requests to Couchdb is around 1000 per sec. I could not increase it further... and getting CouchDB errors like ("mochiweb socket server .... acceptor error... accept failed"). Do you have any idea about this ?
>>
>> Thanks,
>> Sajeesh
>>
>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>> To: <us...@couchdb.apache.org>
>> Sent: Tuesday, May 03, 2011 3:06 PM
>> Subject: Re: Queries on CouchDB tests done using Tsung
>>
>>
>> I'm currently doing some performance analysis/engineering for a customer of mine using tsung (not benchmarking couchdb though). To generate load in this magnitude reliably you'll need either more machines generating the load, or doing more requests in each session (even if they are all the same). Using more requests per session vs one request per session has proven to work for me much better - especially when the response time per request is low.
>>
>> You should specify you scenarios according to your usage patterns (e.g. read/write ratio, accessing views etc.). Simply hammering on reading docs e.g. is probably not very meaningful for a real world application.
>>
>> Another thing you should consider is using a proxy/http accelerator like varnish to serve cacheable resources.
>>
>> On 03.05.2011, at 10:31, Sajeesh P N wrote:
>>
>>> Thanks. I am reading different documents in Couchdb as my requests. My aim is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB and reported in Tsung. Is it necessary to have more clients in Tsung to generate more requests/sec or have more requests per session for one tsung client.
>>>
>>> Thanks in advance,
>>> Sajeesh
>>>
>>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>>> To: <us...@couchdb.apache.org>
>>> Sent: Tuesday, May 03, 2011 1:54 PM
>>> Subject: Re: Queries on CouchDB tests done using Tsung
>>>
>>>
>>> Hey,
>>>
>>> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>>>
>>>> Hi,
>>>>
>>>> I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
>>>>
>>>> 1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
>>>
>>> The default for HTTP load testing in Tsung creates a new connection per request by default, IIRC. But it somehow depends on what you want to test.
>>>
>>>> 2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
>>>
>>> I'd say that's hard to answer per se. What kind of requests? Reading documents? Creating? Accessing views? ...
>>>
>>> If you could elaborate a bit more on what you want to test/benchmark, it might be easier to answer.
>>>
>>>
>>> Best
>>>
>>> Sebastian=
>>>
>>>
>>
>>
>>
>
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sajeesh P N <sa...@tataelxsi.co.in>.
I already tried increasing max_connections to 4096, but still errors are
coming after a certain request rate. The max open files in Couchdb server
system is just 1024. Should we have to increase this value ?
How do we increase the ERL_MAX_PORTS ? If we set this env variable, do we
need to set it again for every couchdb restart ? How to see the existing
configuration values?
Thanks,
Sajeesh
----- Original Message -----
From: "Sebastian Cohnen" <se...@googlemail.com>
To: <us...@couchdb.apache.org>
Sent: Tuesday, May 03, 2011 3:42 PM
Subject: Re: Queries on CouchDB tests done using Tsung
You could try
[httpd]
max_connections = 2048
Also you could try to increase ERL_MAX_PORTS
On 03.05.2011, at 12:03, Sajeesh P N wrote:
> Do we need to increase the max connections configuration in Couchdb to
> increase the no. of requests it can handle. I have 3 Tsung clients, should
> I have to do any system related configurations in Couchdb like max open
> files??
>
> I am facing a problem like, even though I used 3 tsung clients or 1
> client, the max no. of requests to Couchdb is around 1000 per sec. I could
> not increase it further... and getting CouchDB errors like ("mochiweb
> socket server .... acceptor error... accept failed"). Do you have any idea
> about this ?
>
> Thanks,
> Sajeesh
>
> ----- Original Message ----- From: "Sebastian Cohnen"
> <se...@googlemail.com>
> To: <us...@couchdb.apache.org>
> Sent: Tuesday, May 03, 2011 3:06 PM
> Subject: Re: Queries on CouchDB tests done using Tsung
>
>
> I'm currently doing some performance analysis/engineering for a customer
> of mine using tsung (not benchmarking couchdb though). To generate load in
> this magnitude reliably you'll need either more machines generating the
> load, or doing more requests in each session (even if they are all the
> same). Using more requests per session vs one request per session has
> proven to work for me much better - especially when the response time per
> request is low.
>
> You should specify you scenarios according to your usage patterns (e.g.
> read/write ratio, accessing views etc.). Simply hammering on reading docs
> e.g. is probably not very meaningful for a real world application.
>
> Another thing you should consider is using a proxy/http accelerator like
> varnish to serve cacheable resources.
>
> On 03.05.2011, at 10:31, Sajeesh P N wrote:
>
>> Thanks. I am reading different documents in Couchdb as my requests. My
>> aim is to get like 1000, 2000, 3000 ... requests per sec handled by
>> CouchDB and reported in Tsung. Is it necessary to have more clients in
>> Tsung to generate more requests/sec or have more requests per session for
>> one tsung client.
>>
>> Thanks in advance,
>> Sajeesh
>>
>> ----- Original Message ----- From: "Sebastian Cohnen"
>> <se...@googlemail.com>
>> To: <us...@couchdb.apache.org>
>> Sent: Tuesday, May 03, 2011 1:54 PM
>> Subject: Re: Queries on CouchDB tests done using Tsung
>>
>>
>> Hey,
>>
>> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>>
>>> Hi,
>>>
>>> I am presently doing CouchDB performance tests using the Tsung tool
>>> keeping the default configurations for CouchDB. I would like to get a
>>> clarity on the following questions:-
>>>
>>> 1) For every requests queried from Tsung, whether CouchDB is creating a
>>> new connection for each?
>>
>> The default for HTTP load testing in Tsung creates a new connection per
>> request by default, IIRC. But it somehow depends on what you want to
>> test.
>>
>>> 2) I would like to increase the no. of requests handled by CouchDB. What
>>> configuration in Couchdb should I modify and how much should be the
>>> value or limit?
>>
>> I'd say that's hard to answer per se. What kind of requests? Reading
>> documents? Creating? Accessing views? ...
>>
>> If you could elaborate a bit more on what you want to test/benchmark, it
>> might be easier to answer.
>>
>>
>> Best
>>
>> Sebastian=
>>
>>
>
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sebastian Cohnen <se...@googlemail.com>.
You could try
[httpd]
max_connections = 2048
Also you could try to increase ERL_MAX_PORTS
On 03.05.2011, at 12:03, Sajeesh P N wrote:
> Do we need to increase the max connections configuration in Couchdb to increase the no. of requests it can handle. I have 3 Tsung clients, should I have to do any system related configurations in Couchdb like max open files??
>
> I am facing a problem like, even though I used 3 tsung clients or 1 client, the max no. of requests to Couchdb is around 1000 per sec. I could not increase it further... and getting CouchDB errors like ("mochiweb socket server .... acceptor error... accept failed"). Do you have any idea about this ?
>
> Thanks,
> Sajeesh
>
> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
> To: <us...@couchdb.apache.org>
> Sent: Tuesday, May 03, 2011 3:06 PM
> Subject: Re: Queries on CouchDB tests done using Tsung
>
>
> I'm currently doing some performance analysis/engineering for a customer of mine using tsung (not benchmarking couchdb though). To generate load in this magnitude reliably you'll need either more machines generating the load, or doing more requests in each session (even if they are all the same). Using more requests per session vs one request per session has proven to work for me much better - especially when the response time per request is low.
>
> You should specify you scenarios according to your usage patterns (e.g. read/write ratio, accessing views etc.). Simply hammering on reading docs e.g. is probably not very meaningful for a real world application.
>
> Another thing you should consider is using a proxy/http accelerator like varnish to serve cacheable resources.
>
> On 03.05.2011, at 10:31, Sajeesh P N wrote:
>
>> Thanks. I am reading different documents in Couchdb as my requests. My aim is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB and reported in Tsung. Is it necessary to have more clients in Tsung to generate more requests/sec or have more requests per session for one tsung client.
>>
>> Thanks in advance,
>> Sajeesh
>>
>> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
>> To: <us...@couchdb.apache.org>
>> Sent: Tuesday, May 03, 2011 1:54 PM
>> Subject: Re: Queries on CouchDB tests done using Tsung
>>
>>
>> Hey,
>>
>> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>>
>>> Hi,
>>>
>>> I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
>>>
>>> 1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
>>
>> The default for HTTP load testing in Tsung creates a new connection per request by default, IIRC. But it somehow depends on what you want to test.
>>
>>> 2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
>>
>> I'd say that's hard to answer per se. What kind of requests? Reading documents? Creating? Accessing views? ...
>>
>> If you could elaborate a bit more on what you want to test/benchmark, it might be easier to answer.
>>
>>
>> Best
>>
>> Sebastian=
>>
>>
>
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sajeesh P N <sa...@tataelxsi.co.in>.
Do we need to increase the max connections configuration in Couchdb to
increase the no. of requests it can handle. I have 3 Tsung clients, should I
have to do any system related configurations in Couchdb like max open
files??
I am facing a problem like, even though I used 3 tsung clients or 1 client,
the max no. of requests to Couchdb is around 1000 per sec. I could not
increase it further... and getting CouchDB errors like ("mochiweb socket
server .... acceptor error... accept failed"). Do you have any idea about
this ?
Thanks,
Sajeesh
----- Original Message -----
From: "Sebastian Cohnen" <se...@googlemail.com>
To: <us...@couchdb.apache.org>
Sent: Tuesday, May 03, 2011 3:06 PM
Subject: Re: Queries on CouchDB tests done using Tsung
I'm currently doing some performance analysis/engineering for a customer of
mine using tsung (not benchmarking couchdb though). To generate load in this
magnitude reliably you'll need either more machines generating the load, or
doing more requests in each session (even if they are all the same). Using
more requests per session vs one request per session has proven to work for
me much better - especially when the response time per request is low.
You should specify you scenarios according to your usage patterns (e.g.
read/write ratio, accessing views etc.). Simply hammering on reading docs
e.g. is probably not very meaningful for a real world application.
Another thing you should consider is using a proxy/http accelerator like
varnish to serve cacheable resources.
On 03.05.2011, at 10:31, Sajeesh P N wrote:
> Thanks. I am reading different documents in Couchdb as my requests. My aim
> is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB
> and reported in Tsung. Is it necessary to have more clients in Tsung to
> generate more requests/sec or have more requests per session for one tsung
> client.
>
> Thanks in advance,
> Sajeesh
>
> ----- Original Message ----- From: "Sebastian Cohnen"
> <se...@googlemail.com>
> To: <us...@couchdb.apache.org>
> Sent: Tuesday, May 03, 2011 1:54 PM
> Subject: Re: Queries on CouchDB tests done using Tsung
>
>
> Hey,
>
> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>
>> Hi,
>>
>> I am presently doing CouchDB performance tests using the Tsung tool
>> keeping the default configurations for CouchDB. I would like to get a
>> clarity on the following questions:-
>>
>> 1) For every requests queried from Tsung, whether CouchDB is creating a
>> new connection for each?
>
> The default for HTTP load testing in Tsung creates a new connection per
> request by default, IIRC. But it somehow depends on what you want to test.
>
>> 2) I would like to increase the no. of requests handled by CouchDB. What
>> configuration in Couchdb should I modify and how much should be the value
>> or limit?
>
> I'd say that's hard to answer per se. What kind of requests? Reading
> documents? Creating? Accessing views? ...
>
> If you could elaborate a bit more on what you want to test/benchmark, it
> might be easier to answer.
>
>
> Best
>
> Sebastian=
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sebastian Cohnen <se...@googlemail.com>.
I'm currently doing some performance analysis/engineering for a customer of mine using tsung (not benchmarking couchdb though). To generate load in this magnitude reliably you'll need either more machines generating the load, or doing more requests in each session (even if they are all the same). Using more requests per session vs one request per session has proven to work for me much better - especially when the response time per request is low.
You should specify you scenarios according to your usage patterns (e.g. read/write ratio, accessing views etc.). Simply hammering on reading docs e.g. is probably not very meaningful for a real world application.
Another thing you should consider is using a proxy/http accelerator like varnish to serve cacheable resources.
On 03.05.2011, at 10:31, Sajeesh P N wrote:
> Thanks. I am reading different documents in Couchdb as my requests. My aim is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB and reported in Tsung. Is it necessary to have more clients in Tsung to generate more requests/sec or have more requests per session for one tsung client.
>
> Thanks in advance,
> Sajeesh
>
> ----- Original Message ----- From: "Sebastian Cohnen" <se...@googlemail.com>
> To: <us...@couchdb.apache.org>
> Sent: Tuesday, May 03, 2011 1:54 PM
> Subject: Re: Queries on CouchDB tests done using Tsung
>
>
> Hey,
>
> On 03.05.2011, at 09:52, Sajeesh P N wrote:
>
>> Hi,
>>
>> I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
>>
>> 1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
>
> The default for HTTP load testing in Tsung creates a new connection per request by default, IIRC. But it somehow depends on what you want to test.
>
>> 2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
>
> I'd say that's hard to answer per se. What kind of requests? Reading documents? Creating? Accessing views? ...
>
> If you could elaborate a bit more on what you want to test/benchmark, it might be easier to answer.
>
>
> Best
>
> Sebastian=
>
>
Re: Queries on CouchDB tests done using Tsung
Posted by Sajeesh P N <sa...@tataelxsi.co.in>.
Thanks. I am reading different documents in Couchdb as my requests. My aim
is to get like 1000, 2000, 3000 ... requests per sec handled by CouchDB and
reported in Tsung. Is it necessary to have more clients in Tsung to generate
more requests/sec or have more requests per session for one tsung client.
Thanks in advance,
Sajeesh
----- Original Message -----
From: "Sebastian Cohnen" <se...@googlemail.com>
To: <us...@couchdb.apache.org>
Sent: Tuesday, May 03, 2011 1:54 PM
Subject: Re: Queries on CouchDB tests done using Tsung
Hey,
On 03.05.2011, at 09:52, Sajeesh P N wrote:
> Hi,
>
> I am presently doing CouchDB performance tests using the Tsung tool
> keeping the default configurations for CouchDB. I would like to get a
> clarity on the following questions:-
>
> 1) For every requests queried from Tsung, whether CouchDB is creating a
> new connection for each?
The default for HTTP load testing in Tsung creates a new connection per
request by default, IIRC. But it somehow depends on what you want to test.
> 2) I would like to increase the no. of requests handled by CouchDB. What
> configuration in Couchdb should I modify and how much should be the value
> or limit?
I'd say that's hard to answer per se. What kind of requests? Reading
documents? Creating? Accessing views? ...
If you could elaborate a bit more on what you want to test/benchmark, it
might be easier to answer.
Best
Sebastian=
Re: Queries on CouchDB tests done using Tsung
Posted by Sebastian Cohnen <se...@googlemail.com>.
Hey,
On 03.05.2011, at 09:52, Sajeesh P N wrote:
> Hi,
>
> I am presently doing CouchDB performance tests using the Tsung tool keeping the default configurations for CouchDB. I would like to get a clarity on the following questions:-
>
> 1) For every requests queried from Tsung, whether CouchDB is creating a new connection for each?
The default for HTTP load testing in Tsung creates a new connection per request by default, IIRC. But it somehow depends on what you want to test.
> 2) I would like to increase the no. of requests handled by CouchDB. What configuration in Couchdb should I modify and how much should be the value or limit?
I'd say that's hard to answer per se. What kind of requests? Reading documents? Creating? Accessing views? ...
If you could elaborate a bit more on what you want to test/benchmark, it might be easier to answer.
Best
Sebastian