You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Lawrence Pit <la...@gmail.com> on 2009/01/05 05:32:03 UTC
source performance
Hi,
A while back I had installed couchdb v0.8. Today I installed from the
latest source (reporting 0.9.0a731428-incubating); I've set debuglevel
to error. However, when I run the test suite the 0.9 version is 4 to 10
times slower for all tests compared to when I run the test suite using
the 0.8 version.
When installing from source, do I need to provide some extra options so
that it performs optimally?
(didn't find any hints on the wiki)
Both installs are using erlang 5.6.4.
Cheers,
Lawrence
Re: source performance
Posted by Lawrence Pit <la...@gmail.com>.
Geir Magnusson Jr. wrote:
> Well, what he reported for the simple insert test is startling, isn't it?
>
> From 200/sec -> 8/sec between the two versions?
>
> It will be interesting to see if there's a similar slowdown on linux.
> I'm guessing he's on OS X since it's hit the magic number of 8 ("...
> not 9, nor 7, unless of course thou shalt proceed...").
You guessed right, I'm on OS X. ;-)
Cheers,
Lawrence
Re: source performance
Posted by Alan Bell <al...@theopenlearningcentre.com>.
Geir Magnusson Jr. wrote:
>>
>> I do think, that going forward, it would be very cool to have a
>> performance benchmark suite, that was integrated perhaps with Hudson
>> or some other Apache tools. Being able to see our changes effect on
>> performance as we go along will certainly push us in the right
>> direction.
>
> Indeed - I'm actually surprised there isn't something like that already.
>
well there is a test suite with timed results. What is missing is
perhaps a few tests that test specific aspects of performance such as
writing documents, reading them, building lots of view indexes etc. and
a way to share the results. I am thinking maybe the results could be
wrapped up as a json document along with the couchdb version, erlang
version, processor, memory, operating system, disk type and anything
else of interest and posted to a couchdb server somewhere on the web.
> It would be pretty easy to setup. Maybe I'll go poke around on hudson
> a bit and see what I can do.
>
> geir
>
Re: source performance
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Jan 5, 2009, at 1:19 AM, Chris Anderson wrote:
> On Sun, Jan 4, 2009 at 9:20 PM, Lawrence Pit
> <la...@gmail.com> wrote:
>>
>> So for example the test recreate_doc, which hasn't changed for
>> months, takes
>> 345ms using v0.8 and 4406ms using v0.9. That's more than 10 times
>> slower.
>> Same machine, same erlang host.
>
> Quite clearly longer times on the newer version.
>
> A lot has happened since 0.8. I think we were still on inets then, and
> it was before the config module appeared, as well as the move to JSON
> binaries as strings, and some filesystem work. We have not done much
> optimization in the same period, although work is beginning.
>
> There is a new C implementation of the JSON parser on the horizon that
> looks a few times faster, and we've begun to identify some portions of
> the code with low hanging fruit.
>
> I'm not too worried about a performance change over the last few
> months, as we've been concentrating on other things. It's worth noting
> that test suite performance overstates the costs of database creation
> and other setup tasks.
Well, what he reported for the simple insert test is startling, isn't
it?
From 200/sec -> 8/sec between the two versions?
It will be interesting to see if there's a similar slowdown on linux.
I'm guessing he's on OS X since it's hit the magic number of 8 ("...
not 9, nor 7, unless of course thou shalt proceed...").
To me, this is a "broken window" that needs to be looked at. I'd
help, but I don't know the first thing about Erlang :) Who did the
filesystem work?
>
>
> I do think, that going forward, it would be very cool to have a
> performance benchmark suite, that was integrated perhaps with Hudson
> or some other Apache tools. Being able to see our changes effect on
> performance as we go along will certainly push us in the right
> direction.
Indeed - I'm actually surprised there isn't something like that already.
It would be pretty easy to setup. Maybe I'll go poke around on hudson
a bit and see what I can do.
geir
Re: source performance
Posted by Chris Anderson <jc...@gmail.com>.
On Sun, Jan 4, 2009 at 9:20 PM, Lawrence Pit <la...@gmail.com> wrote:
>
> So for example the test recreate_doc, which hasn't changed for months, takes
> 345ms using v0.8 and 4406ms using v0.9. That's more than 10 times slower.
> Same machine, same erlang host.
Quite clearly longer times on the newer version.
A lot has happened since 0.8. I think we were still on inets then, and
it was before the config module appeared, as well as the move to JSON
binaries as strings, and some filesystem work. We have not done much
optimization in the same period, although work is beginning.
There is a new C implementation of the JSON parser on the horizon that
looks a few times faster, and we've begun to identify some portions of
the code with low hanging fruit.
I'm not too worried about a performance change over the last few
months, as we've been concentrating on other things. It's worth noting
that test suite performance overstates the costs of database creation
and other setup tasks.
I do think, that going forward, it would be very cool to have a
performance benchmark suite, that was integrated perhaps with Hudson
or some other Apache tools. Being able to see our changes effect on
performance as we go along will certainly push us in the right
direction.
Chris
--
Chris Anderson
http://jchris.mfdz.com
Re: source performance
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Well, when I asked about the necessity for the then-current level of
commit-to-disk behavior, I got answers like :
On Jan 4, 2009, at 12:45 PM, Jan Lehnardt wrote:
>
> On 4 Jan 2009, at 14:09, Geir Magnusson Jr. wrote:
>>
>> 2) Is anyone using CouchDB in a manner that really requires this
>> level of data security? I appreciate having the *option* to turn
>> on a mode like this, but I don't think I need it all the time. I
>> can use RAID systems that give me battery-backed cache, or I can
>> make the design decision that I am happy to lose X seconds of data
>> in a tradeoff (e.g. do the deep fnctl() every X seconds.....)
>
> Everybody. CouchDB doesn't have an `off`-switch. You just terminate
> the Erlang VM. Without this design, this wouldn't be possible. There
> has been no discussion about optionally making different trade-offs
> for speed and against data integrity, but this is not off-the-map.
So for something this fundamental and stated to be a necessity for
dealing w/ the Erlang VM, it seems odd that it would be prudent to do
a 180-turn and completely invert the behavior w/o any discussion...
I must be missing something.
geir
On Jan 5, 2009, at 4:16 AM, Sven Helmberger wrote:
> Geir Magnusson Jr. schrieb:
>> Doesn't something like this at least warrant a "head up" kind of
>> mail to the dev list? How does it work? How can I control the
>> asynchronicity?
>
> I just understood that as simply responding to the client before you
> really have written everything to disk and queue the writes on the
> server side to be actually written -- which is not something you need
> control over.
>
> Regards,
> Sven Helmberger
Re: source performance
Posted by Sven Helmberger <sv...@gmx.de>.
Geir Magnusson Jr. schrieb:
>
> Doesn't something like this at least warrant a "head up" kind of mail to
> the dev list? How does it work? How can I control the asynchronicity?
>
I just understood that as simply responding to the client before you
really have written everything to disk and queue the writes on the
server side to be actually written -- which is not something you need
control over.
Regards,
Sven Helmberger
Re: source performance
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
I'm confused.
We're in the middle of a discussion on the user list about
performance, I'm told in no uncertain terms that this extreme level of
commit is absolutely required to avoid db corruption because no one
can control what the Erlang VM does (or something...) and without a
word of discussion on the dev list, the behavior is changed?
Doesn't something like this at least warrant a "head up" kind of mail
to the dev list? How does it work? How can I control the
asynchronicity?
geir
On Jan 5, 2009, at 3:58 AM, Jason Davies wrote:
> On 5 Jan 2009, at 08:51, Alan Bell wrote:
>
>> I just upgraded to the SVN code and it got nearly 5 times faster on
>> inserting empty documents I am now up to 390/second on the same
>> hardware that got 80/second before. This machine does recreate_doc
>> in 365ms in the test suite.
>
> I'm assuming the speedup you are seeing is due to the "delayed
> commit" stuff Damien committed in r731452. Apparently documents are
> now committed asynchronously by default.
>
> --
> Jason Davies
>
> www.jasondavies.com
>
Re: source performance
Posted by Jason Davies <ja...@jasondavies.com>.
On 5 Jan 2009, at 08:51, Alan Bell wrote:
> I just upgraded to the SVN code and it got nearly 5 times faster on
> inserting empty documents I am now up to 390/second on the same
> hardware that got 80/second before. This machine does recreate_doc
> in 365ms in the test suite.
I'm assuming the speedup you are seeing is due to the "delayed commit"
stuff Damien committed in r731452. Apparently documents are now
committed asynchronously by default.
--
Jason Davies
www.jasondavies.com
Re: source performance
Posted by Alan Bell <al...@theopenlearningcentre.com>.
I just upgraded to the SVN code and it got nearly 5 times faster on
inserting empty documents I am now up to 390/second on the same hardware
that got 80/second before. This machine does recreate_doc in 365ms in
the test suite.
Server Software: CouchDB/0.9.0a731491-incubating
Server Hostname: 127.0.0.1
Server Port: 5984
Document Path: /test_suite_db_b
Document Length: 70 bytes
Concurrency Level: 10
Time taken for tests: 2.561 seconds
Complete requests: 1000
Failed requests: 249
(Connect: 0, Receive: 0, Length: 249, Exceptions: 0)
Write errors: 0
Total transferred: 281715 bytes
Total POSTed: 152000
HTML transferred: 69715 bytes
Requests per second: 390.53 [#/sec] (mean)
Time per request: 25.606 [ms] (mean)
Time per request: 2.561 [ms] (mean, across all concurrent requests)
Transfer rate: 107.44 [Kbytes/sec] received
57.97 kb/s sent
165.41 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 8
Processing: 8 25 10.6 22 77
Waiting: 7 25 10.2 22 77
Total: 8 25 10.6 22 77
Percentage of the requests served within a certain time (ms)
50% 22
66% 24
75% 27
80% 28
90% 36
95% 52
98% 66
99% 68
100% 77 (longest request)
Lawrence Pit wrote:
>
>>> A while back I had installed couchdb v0.8. Today I installed from
>>> the latest
>>> source (reporting 0.9.0a731428-incubating); I've set debuglevel to
>>> error.
>>> However, when I run the test suite the 0.9 version is 4 to 10 times
>>> slower
>>> for all tests compared to when I run the test suite using the 0.8
>>> version.
>>>
>>
>> There are more tests than in the past, so that should be a factor.
>>
>
> I'm not talking about the total time, I'm referring to the times of
> individual tests. Here's my test run using v0.8:
>
> http://img.skitch.com/20090105-bmutqgq4ag9ba75ynhq6xa69nw.png
>
> and here are the results when using v0.9 source:
>
> http://img.skitch.com/20090105-dfff1t8ybcb1wbj8ebu44p45g1.png
>
> So for example the test recreate_doc, which hasn't changed for months,
> takes 345ms using v0.8 and 4406ms using v0.9. That's more than 10
> times slower. Same machine, same erlang host.
>
>> There is another thread about document insert rate, which shows the
>> ways people are benchmarking CouchDB. If you were able to post some
>> comparative benchmarks of particular operations, so we could see the
>> difference, I'm sure a lot of us would appreciate it
> I'll post my results there, for both couchdb versions.
>
>
>
>
> Cheers,
> Lawrence
>
Re: source performance
Posted by Lawrence Pit <la...@gmail.com>.
>> A while back I had installed couchdb v0.8. Today I installed from the latest
>> source (reporting 0.9.0a731428-incubating); I've set debuglevel to error.
>> However, when I run the test suite the 0.9 version is 4 to 10 times slower
>> for all tests compared to when I run the test suite using the 0.8 version.
>>
>
> There are more tests than in the past, so that should be a factor.
>
I'm not talking about the total time, I'm referring to the times of
individual tests. Here's my test run using v0.8:
http://img.skitch.com/20090105-bmutqgq4ag9ba75ynhq6xa69nw.png
and here are the results when using v0.9 source:
http://img.skitch.com/20090105-dfff1t8ybcb1wbj8ebu44p45g1.png
So for example the test recreate_doc, which hasn't changed for months,
takes 345ms using v0.8 and 4406ms using v0.9. That's more than 10 times
slower. Same machine, same erlang host.
> There is another thread about document insert rate, which shows the
> ways people are benchmarking CouchDB. If you were able to post some
> comparative benchmarks of particular operations, so we could see the
> difference, I'm sure a lot of us would appreciate it
I'll post my results there, for both couchdb versions.
Cheers,
Lawrence
Re: source performance
Posted by Michael McDaniel <co...@autosys.us>.
Regarding Erlang documentation ...
http://erlang.org
Documentation online or you can download source and/or docs. Compile
options as well as starting arguments for erl are described (and, of
course, all modules).
Windows and 'NIX
~Michael
On Sun, Jan 04, 2009 at 08:42:46PM -0800, Chris Anderson wrote:
> On Sun, Jan 4, 2009 at 8:32 PM, Lawrence Pit <la...@gmail.com> wrote:
> > Hi,
> >
> > A while back I had installed couchdb v0.8. Today I installed from the latest
> > source (reporting 0.9.0a731428-incubating); I've set debuglevel to error.
> > However, when I run the test suite the 0.9 version is 4 to 10 times slower
> > for all tests compared to when I run the test suite using the 0.8 version.
>
> There are more tests than in the past, so that should be a factor.
>
> There is another thread about document insert rate, which shows the
> ways people are benchmarking CouchDB. If you were able to post some
> comparative benchmarks of particular operations, so we could see the
> difference, I'm sure a lot of us would appreciate it.
>
> >
> > When installing from source, do I need to provide some extra options so that
> > it performs optimally?
> >
>
> There do seem to be some better versions of Erlang, compile time
> options etc. Does anyone know where this is best documented?
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
--
Michael McDaniel
Portland, Oregon, USA
http://trip.autosys.us
http://autosys.us
Re: source performance
Posted by Chris Anderson <jc...@gmail.com>.
On Sun, Jan 4, 2009 at 8:32 PM, Lawrence Pit <la...@gmail.com> wrote:
> Hi,
>
> A while back I had installed couchdb v0.8. Today I installed from the latest
> source (reporting 0.9.0a731428-incubating); I've set debuglevel to error.
> However, when I run the test suite the 0.9 version is 4 to 10 times slower
> for all tests compared to when I run the test suite using the 0.8 version.
There are more tests than in the past, so that should be a factor.
There is another thread about document insert rate, which shows the
ways people are benchmarking CouchDB. If you were able to post some
comparative benchmarks of particular operations, so we could see the
difference, I'm sure a lot of us would appreciate it.
>
> When installing from source, do I need to provide some extra options so that
> it performs optimally?
>
There do seem to be some better versions of Erlang, compile time
options etc. Does anyone know where this is best documented?
--
Chris Anderson
http://jchris.mfdz.com