You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Konstantin Cherkasoff <k....@gmail.com> on 2011/10/26 01:03:13 UTC
High latency (40ms) and low request rate (10 rps) under windows
*** crossposted to user@couchdb.apache.org
Hi!
I installed CouchDB 1.1.0 on Windows (Winows 7, 32 bit) and tried to
test performance using ab.exe(Apache Benchmark
http://httpd.apache.org/docs/2.2/programs/ab.html)
ab.exe -k -c 1 -n 1000 -p ab.json -T "application/json"
http://localhost:5984/bench
where ab.json contains the simplest document {"value": "test"}.
This utility do 1000 POSTs with no concurency (-c 1) in single
connection (-k for keepalive).
On my laptop (it's NOT low cost netbook) I got 10 requests per
second.So it is 0.1 second per single request.And CPU and HDD
utilization is actually ZERO.
I was just wondering what exactly CouchDB doing all this time (0.1
second) with single little record?As the utilization of CPU and HDD is
0%, I believe that they are not the bottleneck.
So where is bottleneck?
P.S.All test I did with delayed_commits = false.And I tried
socket_options = [{nodelay, true}]
--
Konstantin Cherkasoff
Re: High latency (40ms) and low request rate (10 rps) under windows
Posted by Konstantin Cherkasoff <k....@gmail.com>.
> CouchDB is smart enough to collect writes from multiple concurrent clients into a "group commit" and
> issue only one fsync to durably store all of them in one go. It's a bit like generating a _bulk_docs request under the hood.
> That's the main reason you see the jump from 10/sec to 300/sec.
Hmm... sounds reasonable.
Thank you.
--
Konstantin Cherkasoff
Re: High latency (40ms) and low request rate (10 rps) under windows
Posted by Adam Kocoloski <ko...@apache.org>.
CouchDB is smart enough to collect writes from multiple concurrent clients into a "group commit" and issue only one fsync to durably store all of them in one go. It's a bit like generating a _bulk_docs request under the hood. That's the main reason you see the jump from 10/sec to 300/sec.
Adam
On Oct 26, 2011, at 7:52 PM, Riyad Kalla wrote:
> Hmm, I see what you mean. If a single connection isnt pegging any of the hardware and multiple connections are a magnitude times faster, I am not sure what would be keeping the single connection from performning faster.
>
> Maybe someone else can hop in with ideas?
>
> R
>
> On Oct 26, 2011, at 4:32 PM, Konstantin Cherkasoff <k....@gmail.com> wrote:
>
>> Hi!
>>
>>> What happens when you set delayed_commits to true?
>> With delayed_commits I got about 150 rps so delayed_commits mode is 15
>> times faster.
>>
>>> With that off you are requiring an fsync (iirc) after every value written and then are at the
>>> mercy at your disks write performance (but not in the sense of burst-write
>>> as these are individual, disparate write requests coming in).
>>
>> Yes, I understand that many small writes and fsyncs can be a
>> bottleneck in this case.
>> But I found that the increase of number of concurrent requests lead to
>> an increase in the request rate.
>> For example (100 concurent requests)
>>> ab -k -t 10 -c 100 -n 1000000 -p ab.json -T "application/json" http://localhost:5984/bench
>> shows about 300 RPS (writes and fsyncs) on the same hardware.
>>
>> So, I suppose that 10 requests per second actually is not the limit
>> for this hard disk.
>> And there may be some other problem.
>>
>>> Also, is this build Couch Single Server from Couchbase or some other Windows build of Couch?
>> I tried Couchbase 1.2.0 and
>> this build https://github.com/downloads/dch/couchdb/setup-couchdb-1.1.0+COUCHDB-1152_otp_R14B03.exe
>>
>>
>> --
>> Konstantin
Re: High latency (40ms) and low request rate (10 rps) under windows
Posted by Riyad Kalla <rk...@gmail.com>.
Hmm, I see what you mean. If a single connection isnt pegging any of the hardware and multiple connections are a magnitude times faster, I am not sure what would be keeping the single connection from performning faster.
Maybe someone else can hop in with ideas?
R
On Oct 26, 2011, at 4:32 PM, Konstantin Cherkasoff <k....@gmail.com> wrote:
> Hi!
>
>> What happens when you set delayed_commits to true?
> With delayed_commits I got about 150 rps so delayed_commits mode is 15
> times faster.
>
>> With that off you are requiring an fsync (iirc) after every value written and then are at the
>> mercy at your disks write performance (but not in the sense of burst-write
>> as these are individual, disparate write requests coming in).
>
> Yes, I understand that many small writes and fsyncs can be a
> bottleneck in this case.
> But I found that the increase of number of concurrent requests lead to
> an increase in the request rate.
> For example (100 concurent requests)
>> ab -k -t 10 -c 100 -n 1000000 -p ab.json -T "application/json" http://localhost:5984/bench
> shows about 300 RPS (writes and fsyncs) on the same hardware.
>
> So, I suppose that 10 requests per second actually is not the limit
> for this hard disk.
> And there may be some other problem.
>
>> Also, is this build Couch Single Server from Couchbase or some other Windows build of Couch?
> I tried Couchbase 1.2.0 and
> this build https://github.com/downloads/dch/couchdb/setup-couchdb-1.1.0+COUCHDB-1152_otp_R14B03.exe
>
>
> --
> Konstantin
Re: High latency (40ms) and low request rate (10 rps) under windows
Posted by Konstantin Cherkasoff <k....@gmail.com>.
Hi!
> What happens when you set delayed_commits to true?
With delayed_commits I got about 150 rps so delayed_commits mode is 15
times faster.
> With that off you are requiring an fsync (iirc) after every value written and then are at the
> mercy at your disks write performance (but not in the sense of burst-write
> as these are individual, disparate write requests coming in).
Yes, I understand that many small writes and fsyncs can be a
bottleneck in this case.
But I found that the increase of number of concurrent requests lead to
an increase in the request rate.
For example (100 concurent requests)
> ab -k -t 10 -c 100 -n 1000000 -p ab.json -T "application/json" http://localhost:5984/bench
shows about 300 RPS (writes and fsyncs) on the same hardware.
So, I suppose that 10 requests per second actually is not the limit
for this hard disk.
And there may be some other problem.
> Also, is this build Couch Single Server from Couchbase or some other Windows build of Couch?
I tried Couchbase 1.2.0 and
this build https://github.com/downloads/dch/couchdb/setup-couchdb-1.1.0+COUCHDB-1152_otp_R14B03.exe
--
Konstantin
Re: High latency (40ms) and low request rate (10 rps) under windows
Posted by Riyad Kalla <rk...@gmail.com>.
Konstantin,
What happens when you set delayed_commits to true? With that off you are
requiring an fsync (iirc) after every value written and then are at the
mercy at your disks write performance (but not in the sense of burst-write
as these are individual, disparate write requests coming in).
Also, is this build Couch Single Server from Couchbase or some other Windows
build of Couch?
-R
On Tue, Oct 25, 2011 at 4:03 PM, Konstantin Cherkasoff <
k.cherkasoff@gmail.com> wrote:
> *** crossposted to user@couchdb.apache.org
>
> Hi!
>
> I installed CouchDB 1.1.0 on Windows (Winows 7, 32 bit) and tried to
> test performance using ab.exe(Apache Benchmark
> http://httpd.apache.org/docs/2.2/programs/ab.html)
> ab.exe -k -c 1 -n 1000 -p ab.json -T "application/json"
> http://localhost:5984/bench
> where ab.json contains the simplest document {"value": "test"}.
> This utility do 1000 POSTs with no concurency (-c 1) in single
> connection (-k for keepalive).
> On my laptop (it's NOT low cost netbook) I got 10 requests per
> second.So it is 0.1 second per single request.And CPU and HDD
> utilization is actually ZERO.
> I was just wondering what exactly CouchDB doing all this time (0.1
> second) with single little record?As the utilization of CPU and HDD is
> 0%, I believe that they are not the bottleneck.
>
> So where is bottleneck?
>
>
> P.S.All test I did with delayed_commits = false.And I tried
> socket_options = [{nodelay, true}]
>
> --
> Konstantin Cherkasoff
>