You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jason Smith <jh...@apache.org> on 2012/02/28 14:17:11 UTC
Please report your indexing speed
Forgive the clean new thread. Hopefully it will not remain so.
If you can, would you please clone https://github.com/jhs/slow_couchdb
And build whatever Erlangs and CouchDB checkouts you see fit, and run
the test. For example:
docs=500000 ./bench.sh small_doc.tpl
That should run the test and, God willing, upload the results to a
couch in the cloud. We should be able to use that information to
identify who you are, whether you are on SSD, what Erlang and Couch
build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Feb 29, 2012, at 18:44 , Jan Lehnardt wrote:
>
> On Feb 29, 2012, at 18:42 , Robert Newson wrote:
>
>> My new test again;
>>
>> Filling db.
>> done
>> server is;
>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>> Building views.
>>
>> real 0m24.973s
>> user 0m0.006s
>> sys 0m0.004s
>> done
>> ~/Source/sandbox $ ./bench.sh
>> Filling db.
>> done
>> server is;
>> {"couchdb":"Welcome","version":"1.2.0"}
>> Building views.
>>
>> real 0m21.925s
>> user 0m0.006s
>> sys 0m0.003s
>> done
>>
>> Important Note: The builds that show *regression* were all using
>> SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
>> significant. Can other people who saw regressions report their
>> spidermonkey version?
>
>
> Good observation!
>
> Except for the linux VPS one that had 1.8.0, all previous reports have been
> on 1.8.5.
On the Mac from earlier, using 1.7.0 via homebrew:
1.1.1: 0m54.203s
1.2.x: 0m55.639s
Cheers
Jan
--
>
> Cheers
> Jan
> --
>
>>
>> B.
>>
>> On 29 February 2012 17:25, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>>>>> I've produced a new script that reproduces the view regression. I
>>>>>> apologize in advance for exposing my awful Bash scripting abilities
>>>>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>>>>> works").
>>>>>>
>>>>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>>>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>>>>
>>>>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>>>>
>>>>>
>>>>> hrm with the same script:
>>>>>
>>>>> 1.2.0 :
>>>>> real 0m23.842s
>>>>> user 0m0.007s
>>>>> sys 0m0.004s
>>>>>
>>>>> 1.1.1:
>>>>> real 0m28.625s
>>>>> user 0m0.008s
>>>>> sys 0m0.005s
>>>>>
>>>>> hw specs :
>>>>> $ uname -a
>>>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>>>> mba 2011: i7 4 GB SSD 256 GB
>>>>>
>>>>>
>>>>> - benoît
>>>>
>>>> Just for my curiosity I tested it in the couchdb based distribution I
>>>> maintain (head, last mochiweb + jiffy) :
>>>>
>>>>
>>>> real 0m19.539s
>>>> user 0m0.007s
>>>> sys 0m0.005s
>>>> done
>>>
>>> I launched the tests a second time and same results:
>>>
>>> 1.1.1 :
>>> real 0m29.155s
>>> user 0m0.009s
>>> sys 0m0.006s
>>>
>>> 1.2.0 :
>>>
>>> real 0m25.466s
>>> user 0m0.008s
>>> sys 0m0.005s
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Feb 29, 2012, at 18:42 , Robert Newson wrote:
> My new test again;
>
> Filling db.
> done
> server is;
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
> Building views.
>
> real 0m24.973s
> user 0m0.006s
> sys 0m0.004s
> done
> ~/Source/sandbox $ ./bench.sh
> Filling db.
> done
> server is;
> {"couchdb":"Welcome","version":"1.2.0"}
> Building views.
>
> real 0m21.925s
> user 0m0.006s
> sys 0m0.003s
> done
>
> Important Note: The builds that show *regression* were all using
> SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
> significant. Can other people who saw regressions report their
> spidermonkey version?
Good observation!
Except for the linux VPS one that had 1.8.0, all previous reports have been
on 1.8.5.
Cheers
Jan
--
>
> B.
>
> On 29 February 2012 17:25, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>>>> I've produced a new script that reproduces the view regression. I
>>>>> apologize in advance for exposing my awful Bash scripting abilities
>>>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>>>> works").
>>>>>
>>>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>>>
>>>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>>>
>>>>
>>>> hrm with the same script:
>>>>
>>>> 1.2.0 :
>>>> real 0m23.842s
>>>> user 0m0.007s
>>>> sys 0m0.004s
>>>>
>>>> 1.1.1:
>>>> real 0m28.625s
>>>> user 0m0.008s
>>>> sys 0m0.005s
>>>>
>>>> hw specs :
>>>> $ uname -a
>>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>>> mba 2011: i7 4 GB SSD 256 GB
>>>>
>>>>
>>>> - benoît
>>>
>>> Just for my curiosity I tested it in the couchdb based distribution I
>>> maintain (head, last mochiweb + jiffy) :
>>>
>>>
>>> real 0m19.539s
>>> user 0m0.007s
>>> sys 0m0.005s
>>> done
>>
>> I launched the tests a second time and same results:
>>
>> 1.1.1 :
>> real 0m29.155s
>> user 0m0.009s
>> sys 0m0.006s
>>
>> 1.2.0 :
>>
>> real 0m25.466s
>> user 0m0.008s
>> sys 0m0.005s
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
My new test again;
Filling db.
done
server is;
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
Building views.
real 0m24.973s
user 0m0.006s
sys 0m0.004s
done
~/Source/sandbox $ ./bench.sh
Filling db.
done
server is;
{"couchdb":"Welcome","version":"1.2.0"}
Building views.
real 0m21.925s
user 0m0.006s
sys 0m0.003s
done
Important Note: The builds that show *regression* were all using
SpiderMonkey 1.7.0. The tests above were 1.8.5. I think that's
significant. Can other people who saw regressions report their
spidermonkey version?
B.
On 29 February 2012 17:25, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>>> I've produced a new script that reproduces the view regression. I
>>>> apologize in advance for exposing my awful Bash scripting abilities
>>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>>> works").
>>>>
>>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>>
>>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>>
>>>
>>> hrm with the same script:
>>>
>>> 1.2.0 :
>>> real 0m23.842s
>>> user 0m0.007s
>>> sys 0m0.004s
>>>
>>> 1.1.1:
>>> real 0m28.625s
>>> user 0m0.008s
>>> sys 0m0.005s
>>>
>>> hw specs :
>>> $ uname -a
>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>> mba 2011: i7 4 GB SSD 256 GB
>>>
>>>
>>> - benoît
>>
>> Just for my curiosity I tested it in the couchdb based distribution I
>> maintain (head, last mochiweb + jiffy) :
>>
>>
>> real 0m19.539s
>> user 0m0.007s
>> sys 0m0.005s
>> done
>
> I launched the tests a second time and same results:
>
> 1.1.1 :
> real 0m29.155s
> user 0m0.009s
> sys 0m0.006s
>
> 1.2.0 :
>
> real 0m25.466s
> user 0m0.008s
> sys 0m0.005s
Re: Please report your indexing speed
Posted by Benoit Chesneau <bc...@gmail.com>.
On Wed, Feb 29, 2012 at 6:25 PM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>>> I've produced a new script that reproduces the view regression. I
>>>> apologize in advance for exposing my awful Bash scripting abilities
>>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>>> works").
>>>>
>>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>>
>>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>>
>>>
>>> hrm with the same script:
>>>
>>> 1.2.0 :
>>> real 0m23.842s
>>> user 0m0.007s
>>> sys 0m0.004s
>>>
>>> 1.1.1:
>>> real 0m28.625s
>>> user 0m0.008s
>>> sys 0m0.005s
>>>
>>> hw specs :
>>> $ uname -a
>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>> mba 2011: i7 4 GB SSD 256 GB
>>>
>>>
>>> - benoît
>>
>> Just for my curiosity I tested it in the couchdb based distribution I
>> maintain (head, last mochiweb + jiffy) :
>>
>>
>> real 0m19.539s
>> user 0m0.007s
>> sys 0m0.005s
>> done
>
> I launched the tests a second time and same results:
>
> 1.1.1 :
> real 0m29.155s
> user 0m0.009s
> sys 0m0.006s
>
> 1.2.0 :
>
> real 0m25.466s
> user 0m0.008s
> sys 0m0.005s
tested with 100000 docs :
1.1.1:
real 1m2.352s
user 0m0.007s
sys 0m0.013s
1.2.0:
real 0m52.005s
user 0m0.009s
sys 0m0.008s
All previous tests were with 1.8.5 . With 1.7.0 on 50000 docs:
1.1.1:
real 2m3.828s
user 0m0.009s
sys 0m0.011s
1.2.0:
real 2m49.243s
user 0m0.009s
sys 0m0.011s
- benoît
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
And one more on a Linux Ubuntu VPS, spinning disk dual core cpu:
1.1.1: 0m54.721s
1.2.x: 0m40.356s
(SpiderMonkey 1.8.0)
On Feb 29, 2012, at 18:25 , Benoit Chesneau wrote:
> On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>>> I've produced a new script that reproduces the view regression. I
>>>> apologize in advance for exposing my awful Bash scripting abilities
>>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>>> works").
>>>>
>>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>>
>>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>>
>>>
>>> hrm with the same script:
>>>
>>> 1.2.0 :
>>> real 0m23.842s
>>> user 0m0.007s
>>> sys 0m0.004s
>>>
>>> 1.1.1:
>>> real 0m28.625s
>>> user 0m0.008s
>>> sys 0m0.005s
>>>
>>> hw specs :
>>> $ uname -a
>>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>>> mba 2011: i7 4 GB SSD 256 GB
>>>
>>>
>>> - benoît
>>
>> Just for my curiosity I tested it in the couchdb based distribution I
>> maintain (head, last mochiweb + jiffy) :
>>
>>
>> real 0m19.539s
>> user 0m0.007s
>> sys 0m0.005s
>> done
>
> I launched the tests a second time and same results:
>
> 1.1.1 :
> real 0m29.155s
> user 0m0.009s
> sys 0m0.006s
>
> 1.2.0 :
>
> real 0m25.466s
> user 0m0.008s
> sys 0m0.005s
Re: Please report your indexing speed
Posted by Benoit Chesneau <bc...@gmail.com>.
On Wed, Feb 29, 2012 at 6:07 PM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>>> I've produced a new script that reproduces the view regression. I
>>> apologize in advance for exposing my awful Bash scripting abilities
>>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>>> works").
>>>
>>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>>> complete the same task in only 72% of the time that 1.2.x takes.
>>>
>>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>>
>>
>> hrm with the same script:
>>
>> 1.2.0 :
>> real 0m23.842s
>> user 0m0.007s
>> sys 0m0.004s
>>
>> 1.1.1:
>> real 0m28.625s
>> user 0m0.008s
>> sys 0m0.005s
>>
>> hw specs :
>> $ uname -a
>> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
>> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
>> mba 2011: i7 4 GB SSD 256 GB
>>
>>
>> - benoît
>
> Just for my curiosity I tested it in the couchdb based distribution I
> maintain (head, last mochiweb + jiffy) :
>
>
> real 0m19.539s
> user 0m0.007s
> sys 0m0.005s
> done
I launched the tests a second time and same results:
1.1.1 :
real 0m29.155s
user 0m0.009s
sys 0m0.006s
1.2.0 :
real 0m25.466s
user 0m0.008s
sys 0m0.005s
Re: Please report your indexing speed
Posted by Benoit Chesneau <bc...@gmail.com>.
On Wed, Feb 29, 2012 at 5:53 PM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
>> I've produced a new script that reproduces the view regression. I
>> apologize in advance for exposing my awful Bash scripting abilities
>> (also my inability to write "pure" shell. Your "Bashism" is my "it
>> works").
>>
>> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
>> complete the same task in only 72% of the time that 1.2.x takes.
>>
>> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>>
>
> hrm with the same script:
>
> 1.2.0 :
> real 0m23.842s
> user 0m0.007s
> sys 0m0.004s
>
> 1.1.1:
> real 0m28.625s
> user 0m0.008s
> sys 0m0.005s
>
> hw specs :
> $ uname -a
> Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
> 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
> mba 2011: i7 4 GB SSD 256 GB
>
>
> - benoît
Just for my curiosity I tested it in the couchdb based distribution I
maintain (head, last mochiweb + jiffy) :
real 0m19.539s
user 0m0.007s
sys 0m0.005s
done
Re: Please report your indexing speed
Posted by Benoit Chesneau <bc...@gmail.com>.
On Wed, Feb 29, 2012 at 4:03 PM, Robert Newson <rn...@apache.org> wrote:
> I've produced a new script that reproduces the view regression. I
> apologize in advance for exposing my awful Bash scripting abilities
> (also my inability to write "pure" shell. Your "Bashism" is my "it
> works").
>
> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
> complete the same task in only 72% of the time that 1.2.x takes.
>
> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>
hrm with the same script:
1.2.0 :
real 0m23.842s
user 0m0.007s
sys 0m0.004s
1.1.1:
real 0m28.625s
user 0m0.008s
sys 0m0.005s
hw specs :
$ uname -a
Darwin enki.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12
18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
mba 2011: i7 4 GB SSD 256 GB
- benoît
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
That same time but for 10x doc count;
1.1.x: 10m3.734s
1.2.x: 14m17.877s
My machine is a Macbook Air 1.7 GHz Intel Core i5 with 4 GB 1333 MHz
DDR3 running OS X 10.7.3.
On 29 February 2012 15:03, Robert Newson <rn...@apache.org> wrote:
> I've produced a new script that reproduces the view regression. I
> apologize in advance for exposing my awful Bash scripting abilities
> (also my inability to write "pure" shell. Your "Bashism" is my "it
> works").
>
> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
> complete the same task in only 72% of the time that 1.2.x takes.
>
> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>
> It would be very useful if others could reproduce (or contradict!) this finding.
>
> B.
>
> On 29 February 2012 00:56, Jan Lehnardt <ja...@apache.org> wrote:
>> One more report.
>>
>> I got suspicious of the rather short runtimes, so I picked the
>> default_doc and ran it at 500k:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s
>> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s
>>
>> It seems to me that we need more variation in what we test,
>> more OSs, larger ddocs, like the one Stefan linked to. Can
>> anyone help providing this?
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
>>
>>> For Robert Newson, avoiding bulk inserts to populate the dbs:
>>>
>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s
>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s
>>>
>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s
>>>
>>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s
>>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s
>>>
>>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s
>>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s
>>>
>>> (Just R14B04, full logs available on demand)
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>>>
>>>> Same story, but spinning disk, 5400rpm:
>>>>
>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
>>>>
>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
>>>>
>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
>>>>
>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s
>>>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s
>>>>
>>>> Full logs at http://jan.prima.de/slow_couch/rust/
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>>>
>>>>>
>>>>> # tl;dr:
>>>>>
>>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
>>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
>>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
>>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>>>>>
>>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
>>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
>>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
>>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>>>>>
>>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
>>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
>>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
>>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>>>>>
>>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
>>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
>>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
>>>>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>>>>>
>>>>> (Full logs at [5])
>>>>>
>>>>>
>>>>> # Description
>>>>>
>>>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>>>
>>>>> I'll be running the same set on spinning disk and then Robert N asked
>>>>> me to populate the DBs not using builk docs. Since that's gonna take
>>>>> a while, I'll probably run this overnight.
>>>>>
>>>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>>>> and Filipe's seatoncouch[2].
>>>>>
>>>>> The changes I've made is have the small_doc test run with 500k instead
>>>>> of 50k docs, added .view files to match the tpl files in
>>>>> seatoncouch/templates/* so we can have similar views use the different
>>>>> doc structures.
>>>>>
>>>>> I also added two scripts to orchestrate the above testing in a more
>>>>> automated fashion. It also allows you to run the full matrix yourself.
>>>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>>>> there as well).
>>>>>
>>>>> matrix.sh also allows you to specify which docs to run.
>>>>>
>>>>> Please shout if you need any more info about this test run or how to
>>>>> run this yourself.
>>>>>
>>>>>
>>>>> # Analysis
>>>>>
>>>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>>>> there are more differences that shake out and I've yet have to run this
>>>>> on a spinning disk let alone another OS* or more complex view functions
>>>>> or larger design docs (like the one Stefan had).
>>>>>
>>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>>>> definitely be looking into that, but we can do with every bit of help :)
>>>>>
>>>>> So far, I'm happy to conclude that while there are definitely provable
>>>>> differences, that we can live with them.
>>>>>
>>>>> Cheers
>>>>> Jan
>>>>> --
>>>>>
>>>>>
>>>>> [1]: https://github.com/janl/slow_couchdb
>>>>> [2]: https://github.com/janl/seatoncouch
>>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>>>
>>>>>
>>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>>>
>>>>>> Jason, repeated my last test with the 1Kb docs (
>>>>>> https://gist.github.com/1930804, map function
>>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>>>> as well. Here are the full results (also in
>>>>>> https://gist.github.com/1930807):
>>>>>>
>>>>>>
>>>>>> Before COUCHDB-1186
>>>>>>
>>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m6.676s
>>>>>> user 0m0.009s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> After COUCHDB-1186
>>>>>>
>>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m1.395s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> After COUCHDB-1186 + better queueing patch
>>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>>>
>>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 4m48.175s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.009s
>>>>>>
>>>>>>
>>>>>> CouchDB branch 1.1.x
>>>>>>
>>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m44.374s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>>
>>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>>
>>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>>>> the test. For example:
>>>>>>>
>>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>>
>>>>>>> That should run the test and, God willing, upload the results to a
>>>>>>> couch in the cloud. We should be able to use that information to
>>>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Filipe David Manana,
>>>>>>
>>>>>> "Reasonable men adapt themselves to the world.
>>>>>> Unreasonable men adapt the world to themselves.
>>>>>> That's why all progress depends on unreasonable men."
>>>>>
>>>>
>>>
>>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
Mac OS X 10.7.3, 2.6Ghz i7 (2x2 cores), R14B04
SSD:
1.1.1: 0m31.337s
1.2.x: 0m21.229s
Spinning disk, 5400rpm:
1.1.1: 0m32.807s
1.2.x: 0m21.172s
Cheers
Jan
--
On Feb 29, 2012, at 16:03 , Robert Newson wrote:
> I've produced a new script that reproduces the view regression. I
> apologize in advance for exposing my awful Bash scripting abilities
> (also my inability to write "pure" shell. Your "Bashism" is my "it
> works").
>
> I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
> complete the same task in only 72% of the time that 1.2.x takes.
>
> http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
>
> It would be very useful if others could reproduce (or contradict!) this finding.
>
> B.
>
> On 29 February 2012 00:56, Jan Lehnardt <ja...@apache.org> wrote:
>> One more report.
>>
>> I got suspicious of the rather short runtimes, so I picked the
>> default_doc and ran it at 500k:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s
>> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s
>>
>> It seems to me that we need more variation in what we test,
>> more OSs, larger ddocs, like the one Stefan linked to. Can
>> anyone help providing this?
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
>>
>>> For Robert Newson, avoiding bulk inserts to populate the dbs:
>>>
>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s
>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s
>>>
>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s
>>>
>>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s
>>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s
>>>
>>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s
>>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s
>>>
>>> (Just R14B04, full logs available on demand)
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>>>
>>>> Same story, but spinning disk, 5400rpm:
>>>>
>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
>>>>
>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
>>>>
>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
>>>>
>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s
>>>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s
>>>>
>>>> Full logs at http://jan.prima.de/slow_couch/rust/
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>>>
>>>>>
>>>>> # tl;dr:
>>>>>
>>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
>>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
>>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
>>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>>>>>
>>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
>>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
>>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
>>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>>>>>
>>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
>>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
>>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
>>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>>>>>
>>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
>>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
>>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
>>>>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>>>>>
>>>>> (Full logs at [5])
>>>>>
>>>>>
>>>>> # Description
>>>>>
>>>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>>>
>>>>> I'll be running the same set on spinning disk and then Robert N asked
>>>>> me to populate the DBs not using builk docs. Since that's gonna take
>>>>> a while, I'll probably run this overnight.
>>>>>
>>>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>>>> and Filipe's seatoncouch[2].
>>>>>
>>>>> The changes I've made is have the small_doc test run with 500k instead
>>>>> of 50k docs, added .view files to match the tpl files in
>>>>> seatoncouch/templates/* so we can have similar views use the different
>>>>> doc structures.
>>>>>
>>>>> I also added two scripts to orchestrate the above testing in a more
>>>>> automated fashion. It also allows you to run the full matrix yourself.
>>>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>>>> there as well).
>>>>>
>>>>> matrix.sh also allows you to specify which docs to run.
>>>>>
>>>>> Please shout if you need any more info about this test run or how to
>>>>> run this yourself.
>>>>>
>>>>>
>>>>> # Analysis
>>>>>
>>>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>>>> there are more differences that shake out and I've yet have to run this
>>>>> on a spinning disk let alone another OS* or more complex view functions
>>>>> or larger design docs (like the one Stefan had).
>>>>>
>>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>>>> definitely be looking into that, but we can do with every bit of help :)
>>>>>
>>>>> So far, I'm happy to conclude that while there are definitely provable
>>>>> differences, that we can live with them.
>>>>>
>>>>> Cheers
>>>>> Jan
>>>>> --
>>>>>
>>>>>
>>>>> [1]: https://github.com/janl/slow_couchdb
>>>>> [2]: https://github.com/janl/seatoncouch
>>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>>>
>>>>>
>>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>>>
>>>>>> Jason, repeated my last test with the 1Kb docs (
>>>>>> https://gist.github.com/1930804, map function
>>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>>>> as well. Here are the full results (also in
>>>>>> https://gist.github.com/1930807):
>>>>>>
>>>>>>
>>>>>> Before COUCHDB-1186
>>>>>>
>>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m6.676s
>>>>>> user 0m0.009s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> After COUCHDB-1186
>>>>>>
>>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m1.395s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> After COUCHDB-1186 + better queueing patch
>>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>>>
>>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>>> ]}
>>>>>>
>>>>>> real 4m48.175s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.009s
>>>>>>
>>>>>>
>>>>>> CouchDB branch 1.1.x
>>>>>>
>>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>>> batch=5000 ./bench.sh wow.tpl
>>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>>>
>>>>>> [INFO] Created DB named `db1'
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> (....)
>>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>>> Building view.
>>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>>>> ]}
>>>>>>
>>>>>> real 5m44.374s
>>>>>> user 0m0.008s
>>>>>> sys 0m0.010s
>>>>>>
>>>>>>
>>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>>
>>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>>
>>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>>>> the test. For example:
>>>>>>>
>>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>>
>>>>>>> That should run the test and, God willing, upload the results to a
>>>>>>> couch in the cloud. We should be able to use that information to
>>>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Filipe David Manana,
>>>>>>
>>>>>> "Reasonable men adapt themselves to the world.
>>>>>> Unreasonable men adapt the world to themselves.
>>>>>> That's why all progress depends on unreasonable men."
>>>>>
>>>>
>>>
>>
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
I've produced a new script that reproduces the view regression. I
apologize in advance for exposing my awful Bash scripting abilities
(also my inability to write "pure" shell. Your "Bashism" is my "it
works").
I get 0m56.521s for 1.1.x and 1m17.108s for 1.2.x. That is, 1.1.x
complete the same task in only 72% of the time that 1.2.x takes.
http://friendpaste.com/UI1OcECLEzR6i4D75LqQy
It would be very useful if others could reproduce (or contradict!) this finding.
B.
On 29 February 2012 00:56, Jan Lehnardt <ja...@apache.org> wrote:
> One more report.
>
> I got suspicious of the rather short runtimes, so I picked the
> default_doc and ran it at 500k:
>
> bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s
> bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s
>
> It seems to me that we need more variation in what we test,
> more OSs, larger ddocs, like the one Stefan linked to. Can
> anyone help providing this?
>
> Cheers
> Jan
> --
>
>
> On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
>
>> For Robert Newson, avoiding bulk inserts to populate the dbs:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s
>> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s
>>
>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s
>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s
>>
>> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s
>> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s
>>
>> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s
>> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s
>>
>> (Just R14B04, full logs available on demand)
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>>
>>> Same story, but spinning disk, 5400rpm:
>>>
>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
>>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
>>>
>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
>>>
>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
>>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
>>>
>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
>>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s
>>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s
>>>
>>> Full logs at http://jan.prima.de/slow_couch/rust/
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>>
>>>>
>>>> # tl;dr:
>>>>
>>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
>>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
>>>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
>>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>>>>
>>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
>>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
>>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
>>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>>>>
>>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
>>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
>>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
>>>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>>>>
>>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
>>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
>>>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
>>>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>>>>
>>>> (Full logs at [5])
>>>>
>>>>
>>>> # Description
>>>>
>>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>>
>>>> I'll be running the same set on spinning disk and then Robert N asked
>>>> me to populate the DBs not using builk docs. Since that's gonna take
>>>> a while, I'll probably run this overnight.
>>>>
>>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>>> and Filipe's seatoncouch[2].
>>>>
>>>> The changes I've made is have the small_doc test run with 500k instead
>>>> of 50k docs, added .view files to match the tpl files in
>>>> seatoncouch/templates/* so we can have similar views use the different
>>>> doc structures.
>>>>
>>>> I also added two scripts to orchestrate the above testing in a more
>>>> automated fashion. It also allows you to run the full matrix yourself.
>>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>>> there as well).
>>>>
>>>> matrix.sh also allows you to specify which docs to run.
>>>>
>>>> Please shout if you need any more info about this test run or how to
>>>> run this yourself.
>>>>
>>>>
>>>> # Analysis
>>>>
>>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>>> there are more differences that shake out and I've yet have to run this
>>>> on a spinning disk let alone another OS* or more complex view functions
>>>> or larger design docs (like the one Stefan had).
>>>>
>>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>>> definitely be looking into that, but we can do with every bit of help :)
>>>>
>>>> So far, I'm happy to conclude that while there are definitely provable
>>>> differences, that we can live with them.
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>>
>>>> [1]: https://github.com/janl/slow_couchdb
>>>> [2]: https://github.com/janl/seatoncouch
>>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>>
>>>>
>>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>>
>>>>> Jason, repeated my last test with the 1Kb docs (
>>>>> https://gist.github.com/1930804, map function
>>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>>> as well. Here are the full results (also in
>>>>> https://gist.github.com/1930807):
>>>>>
>>>>>
>>>>> Before COUCHDB-1186
>>>>>
>>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real 5m6.676s
>>>>> user 0m0.009s
>>>>> sys 0m0.010s
>>>>>
>>>>>
>>>>> After COUCHDB-1186
>>>>>
>>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real 5m1.395s
>>>>> user 0m0.008s
>>>>> sys 0m0.010s
>>>>>
>>>>>
>>>>> After COUCHDB-1186 + better queueing patch
>>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>>
>>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>>> ]}
>>>>>
>>>>> real 4m48.175s
>>>>> user 0m0.008s
>>>>> sys 0m0.009s
>>>>>
>>>>>
>>>>> CouchDB branch 1.1.x
>>>>>
>>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>>> batch=5000 ./bench.sh wow.tpl
>>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>>
>>>>> [INFO] Created DB named `db1'
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> (....)
>>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>>> Building view.
>>>>> {"total_rows":500000,"offset":0,"rows":[
>>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>>> ]}
>>>>>
>>>>> real 5m44.374s
>>>>> user 0m0.008s
>>>>> sys 0m0.010s
>>>>>
>>>>>
>>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>
>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>
>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>>> the test. For example:
>>>>>>
>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>
>>>>>> That should run the test and, God willing, upload the results to a
>>>>>> couch in the cloud. We should be able to use that information to
>>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Filipe David Manana,
>>>>>
>>>>> "Reasonable men adapt themselves to the world.
>>>>> Unreasonable men adapt the world to themselves.
>>>>> That's why all progress depends on unreasonable men."
>>>>
>>>
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
One more report.
I got suspicious of the rather short runtimes, so I picked the
default_doc and ran it at 500k:
bench_R14B04_1.1.1_default_doc.tpl.log 2m19.139s
bench_R14B04_1.2.x_default_doc.tpl.log 2m18.875s
It seems to me that we need more variation in what we test,
more OSs, larger ddocs, like the one Stefan linked to. Can
anyone help providing this?
Cheers
Jan
--
On Feb 29, 2012, at 00:23 , Jan Lehnardt wrote:
> For Robert Newson, avoiding bulk inserts to populate the dbs:
>
> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s
> bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s
>
> bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s
> bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s
>
> bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s
> bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s
>
> bench_R14B04_1.1.1_wow.tpl.log 0m33.944s
> bench_R14B04_1.2.x_wow.tpl.log 0m27.087s
>
> (Just R14B04, full logs available on demand)
>
> Cheers
> Jan
> --
>
>
> On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
>
>> Same story, but spinning disk, 5400rpm:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
>> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
>>
>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
>> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
>> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
>>
>> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
>> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
>> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
>>
>> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
>> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
>> bench_R15B_1.1.1_wow.tpl.log 0m20.171s
>> bench_R15B_1.2.x_wow.tpl.log 0m18.800s
>>
>> Full logs at http://jan.prima.de/slow_couch/rust/
>>
>> Cheers
>> Jan
>> --
>>
>>
>> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>>
>>>
>>> # tl;dr:
>>>
>>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
>>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
>>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
>>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>>>
>>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
>>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
>>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
>>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>>>
>>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
>>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
>>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
>>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>>>
>>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
>>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
>>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
>>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>>>
>>> (Full logs at [5])
>>>
>>>
>>> # Description
>>>
>>> All of these are on Mac OS X 10.7.3 on an SSD.
>>>
>>> I'll be running the same set on spinning disk and then Robert N asked
>>> me to populate the DBs not using builk docs. Since that's gonna take
>>> a while, I'll probably run this overnight.
>>>
>>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>>> and Filipe's seatoncouch[2].
>>>
>>> The changes I've made is have the small_doc test run with 500k instead
>>> of 50k docs, added .view files to match the tpl files in
>>> seatoncouch/templates/* so we can have similar views use the different
>>> doc structures.
>>>
>>> I also added two scripts to orchestrate the above testing in a more
>>> automated fashion. It also allows you to run the full matrix yourself.
>>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>>> there as well).
>>>
>>> matrix.sh also allows you to specify which docs to run.
>>>
>>> Please shout if you need any more info about this test run or how to
>>> run this yourself.
>>>
>>>
>>> # Analysis
>>>
>>> Inconclusive, I'l like to run this on larger dbs in general to see if
>>> there are more differences that shake out and I've yet have to run this
>>> on a spinning disk let alone another OS* or more complex view functions
>>> or larger design docs (like the one Stefan had).
>>>
>>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>>> definitely be looking into that, but we can do with every bit of help :)
>>>
>>> So far, I'm happy to conclude that while there are definitely provable
>>> differences, that we can live with them.
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>> [1]: https://github.com/janl/slow_couchdb
>>> [2]: https://github.com/janl/seatoncouch
>>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>>> [5]: http://jan.prima.de/slow_couch/ssd/
>>>
>>>
>>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>>
>>>> Jason, repeated my last test with the 1Kb docs (
>>>> https://gist.github.com/1930804, map function
>>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>>> as well. Here are the full results (also in
>>>> https://gist.github.com/1930807):
>>>>
>>>>
>>>> Before COUCHDB-1186
>>>>
>>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>>
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>>
>>>> real 5m6.676s
>>>> user 0m0.009s
>>>> sys 0m0.010s
>>>>
>>>>
>>>> After COUCHDB-1186
>>>>
>>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>>
>>>> real 5m1.395s
>>>> user 0m0.008s
>>>> sys 0m0.010s
>>>>
>>>>
>>>> After COUCHDB-1186 + better queueing patch
>>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>>
>>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>>
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>>> ]}
>>>>
>>>> real 4m48.175s
>>>> user 0m0.008s
>>>> sys 0m0.009s
>>>>
>>>>
>>>> CouchDB branch 1.1.x
>>>>
>>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>>> batch=5000 ./bench.sh wow.tpl
>>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>>
>>>> [INFO] Created DB named `db1'
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> (....)
>>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>>> Building view.
>>>> {"total_rows":500000,"offset":0,"rows":[
>>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>>> ]}
>>>>
>>>> real 5m44.374s
>>>> user 0m0.008s
>>>> sys 0m0.010s
>>>>
>>>>
>>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>>
>>>>
>>>>
>>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>
>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>
>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>> the test. For example:
>>>>>
>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>
>>>>> That should run the test and, God willing, upload the results to a
>>>>> couch in the cloud. We should be able to use that information to
>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>> build, and how fast it ran. Modulo bugs.
>>>>
>>>>
>>>>
>>>> --
>>>> Filipe David Manana,
>>>>
>>>> "Reasonable men adapt themselves to the world.
>>>> Unreasonable men adapt the world to themselves.
>>>> That's why all progress depends on unreasonable men."
>>>
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
For Robert Newson, avoiding bulk inserts to populate the dbs:
bench_R14B04_1.1.1_default_doc.tpl.log 0m19.692s
bench_R14B04_1.2.x_default_doc.tpl.log 0m17.033s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m31.393s
bench_R14B04_1.2.x_nested_6k.tpl.log 0m42.010s
bench_R14B04_1.1.1_small_doc.tpl.log 0m8.103s
bench_R14B04_1.2.x_small_doc.tpl.log 0m10.597s
bench_R14B04_1.1.1_wow.tpl.log 0m33.944s
bench_R14B04_1.2.x_wow.tpl.log 0m27.087s
(Just R14B04, full logs available on demand)
Cheers
Jan
--
On Feb 28, 2012, at 23:09 , Jan Lehnardt wrote:
> Same story, but spinning disk, 5400rpm:
>
> bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
> bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
> bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
>
> bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
> bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
> bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
> bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
>
> bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
> bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
> bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
> bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
>
> bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
> bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
> bench_R15B_1.1.1_wow.tpl.log 0m20.171s
> bench_R15B_1.2.x_wow.tpl.log 0m18.800s
>
> Full logs at http://jan.prima.de/slow_couch/rust/
>
> Cheers
> Jan
> --
>
>
> On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>
>>
>> # tl;dr:
>>
>> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
>> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
>> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
>> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>>
>> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
>> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
>> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
>> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>>
>> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
>> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
>> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
>> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>>
>> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
>> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
>> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
>> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>>
>> (Full logs at [5])
>>
>>
>> # Description
>>
>> All of these are on Mac OS X 10.7.3 on an SSD.
>>
>> I'll be running the same set on spinning disk and then Robert N asked
>> me to populate the DBs not using builk docs. Since that's gonna take
>> a while, I'll probably run this overnight.
>>
>> All of the results are generated by my fork of Jason's slow_couchdb[1]
>> and Filipe's seatoncouch[2].
>>
>> The changes I've made is have the small_doc test run with 500k instead
>> of 50k docs, added .view files to match the tpl files in
>> seatoncouch/templates/* so we can have similar views use the different
>> doc structures.
>>
>> I also added two scripts to orchestrate the above testing in a more
>> automated fashion. It also allows you to run the full matrix yourself.
>> All you need is set up homebrew allow `brew switch erlang R14B04` and
>> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
>> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
>> (which is controlled in runner.sh[4], adjust the path to the git checkout
>> there as well).
>>
>> matrix.sh also allows you to specify which docs to run.
>>
>> Please shout if you need any more info about this test run or how to
>> run this yourself.
>>
>>
>> # Analysis
>>
>> Inconclusive, I'l like to run this on larger dbs in general to see if
>> there are more differences that shake out and I've yet have to run this
>> on a spinning disk let alone another OS* or more complex view functions
>> or larger design docs (like the one Stefan had).
>>
>> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
>> definitely be looking into that, but we can do with every bit of help :)
>>
>> So far, I'm happy to conclude that while there are definitely provable
>> differences, that we can live with them.
>>
>> Cheers
>> Jan
>> --
>>
>>
>> [1]: https://github.com/janl/slow_couchdb
>> [2]: https://github.com/janl/seatoncouch
>> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
>> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
>> [5]: http://jan.prima.de/slow_couch/ssd/
>>
>>
>> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>>
>>> Jason, repeated my last test with the 1Kb docs (
>>> https://gist.github.com/1930804, map function
>>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>>> as well. Here are the full results (also in
>>> https://gist.github.com/1930807):
>>>
>>>
>>> Before COUCHDB-1186
>>>
>>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":500000,"offset":0,"rows":[
>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>> ]}
>>>
>>> real 5m6.676s
>>> user 0m0.009s
>>> sys 0m0.010s
>>>
>>>
>>> After COUCHDB-1186
>>>
>>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":500000,"offset":0,"rows":[
>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>> ]}
>>>
>>> real 5m1.395s
>>> user 0m0.008s
>>> sys 0m0.010s
>>>
>>>
>>> After COUCHDB-1186 + better queueing patch
>>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>>
>>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":500000,"offset":0,"rows":[
>>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>>> ]}
>>>
>>> real 4m48.175s
>>> user 0m0.008s
>>> sys 0m0.009s
>>>
>>>
>>> CouchDB branch 1.1.x
>>>
>>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":500000,"offset":0,"rows":[
>>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>>> ]}
>>>
>>> real 5m44.374s
>>> user 0m0.008s
>>> sys 0m0.010s
>>>
>>>
>>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>>
>>>
>>>
>>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>
>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>
>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>> the test. For example:
>>>>
>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>
>>>> That should run the test and, God willing, upload the results to a
>>>> couch in the cloud. We should be able to use that information to
>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>> build, and how fast it ran. Modulo bugs.
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
Same story, but spinning disk, 5400rpm:
bench_R14B04_1.1.1_default_doc.tpl.log 0m19.175s
bench_R14B04_1.2.x_default_doc.tpl.log 0m16.821s
bench_R15B_1.1.1_default_doc.tpl.log 0m13.050s
bench_R15B_1.2.x_default_doc.tpl.log 0m13.292s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m26.941s
bench_R14B04_1.2.x_nested_6k.tpl.log 0m39.178s
bench_R15B_1.1.1_nested_6k.tpl.log 0m47.766s
bench_R15B_1.2.x_nested_6k.tpl.log 0m31.697s
bench_R14B04_1.1.1_small_doc.tpl.log 1m19.851s
bench_R14B04_1.2.x_small_doc.tpl.log 1m43.057s
bench_R15B_1.1.1_small_doc.tpl.log 0m52.249s
bench_R15B_1.2.x_small_doc.tpl.log 1m8.195s
bench_R14B04_1.1.1_wow.tpl.log 0m29.589s
bench_R14B04_1.2.x_wow.tpl.log 0m24.867s
bench_R15B_1.1.1_wow.tpl.log 0m20.171s
bench_R15B_1.2.x_wow.tpl.log 0m18.800s
Full logs at http://jan.prima.de/slow_couch/rust/
Cheers
Jan
--
On Feb 28, 2012, at 21:22 , Jan Lehnardt wrote:
>
> # tl;dr:
>
> bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
> bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
> bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
> bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
>
> bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
> bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
> bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
> bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
>
> bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
> bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
> bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
> bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
>
> bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
> bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
> bench_R15B_1.1.1_wow.tpl.log 0m20.493s
> bench_R15B_1.2.x_wow.tpl.log 0m19.584s
>
> (Full logs at [5])
>
>
> # Description
>
> All of these are on Mac OS X 10.7.3 on an SSD.
>
> I'll be running the same set on spinning disk and then Robert N asked
> me to populate the DBs not using builk docs. Since that's gonna take
> a while, I'll probably run this overnight.
>
> All of the results are generated by my fork of Jason's slow_couchdb[1]
> and Filipe's seatoncouch[2].
>
> The changes I've made is have the small_doc test run with 500k instead
> of 50k docs, added .view files to match the tpl files in
> seatoncouch/templates/* so we can have similar views use the different
> doc structures.
>
> I also added two scripts to orchestrate the above testing in a more
> automated fashion. It also allows you to run the full matrix yourself.
> All you need is set up homebrew allow `brew switch erlang R14B04` and
> R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
> CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
> (which is controlled in runner.sh[4], adjust the path to the git checkout
> there as well).
>
> matrix.sh also allows you to specify which docs to run.
>
> Please shout if you need any more info about this test run or how to
> run this yourself.
>
>
> # Analysis
>
> Inconclusive, I'l like to run this on larger dbs in general to see if
> there are more differences that shake out and I've yet have to run this
> on a spinning disk let alone another OS* or more complex view functions
> or larger design docs (like the one Stefan had).
>
> * It shouldn't be too much work to port slow_couchdb to other OSs, I'll
> definitely be looking into that, but we can do with every bit of help :)
>
> So far, I'm happy to conclude that while there are definitely provable
> differences, that we can live with them.
>
> Cheers
> Jan
> --
>
>
> [1]: https://github.com/janl/slow_couchdb
> [2]: https://github.com/janl/seatoncouch
> [3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
> [4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
> [5]: http://jan.prima.de/slow_couch/ssd/
>
>
> On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
>
>> Jason, repeated my last test with the 1Kb docs (
>> https://gist.github.com/1930804, map function
>> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
>> as well. Here are the full results (also in
>> https://gist.github.com/1930807):
>>
>>
>> Before COUCHDB-1186
>>
>> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>> ]}
>>
>> real 5m6.676s
>> user 0m0.009s
>> sys 0m0.010s
>>
>>
>> After COUCHDB-1186
>>
>> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>> ]}
>>
>> real 5m1.395s
>> user 0m0.008s
>> sys 0m0.010s
>>
>>
>> After COUCHDB-1186 + better queueing patch
>> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>>
>> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
>> ]}
>>
>> real 4m48.175s
>> user 0m0.008s
>> sys 0m0.009s
>>
>>
>> CouchDB branch 1.1.x
>>
>> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":500000,"offset":0,"rows":[
>> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
>> ]}
>>
>> real 5m44.374s
>> user 0m0.008s
>> sys 0m0.010s
>>
>>
>> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>>
>>
>>
>> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
# tl;dr:
bench_R14B04_1.1.1_default_doc.tpl.log 0m18.749s
bench_R14B04_1.2.x_default_doc.tpl.log 0m16.304s
bench_R15B_1.1.1_default_doc.tpl.log 0m12.946s
bench_R15B_1.2.x_default_doc.tpl.log 0m13.616s
bench_R14B04_1.1.1_nested_6k.tpl.log 1m27.267s
bench_R14B04_1.2.x_nested_6k.tpl.log 0m37.910s
bench_R15B_1.1.1_nested_6k.tpl.log 0m46.963s
bench_R15B_1.2.x_nested_6k.tpl.log 0m33.011s
bench_R14B04_1.1.1_small_doc.tpl.log 1m17.212s
bench_R14B04_1.2.x_small_doc.tpl.log 1m41.383s
bench_R15B_1.1.1_small_doc.tpl.log 0m52.858s
bench_R15B_1.2.x_small_doc.tpl.log 1m9.043s
bench_R14B04_1.1.1_wow.tpl.log 0m29.842s
bench_R14B04_1.2.x_wow.tpl.log 0m24.178s
bench_R15B_1.1.1_wow.tpl.log 0m20.493s
bench_R15B_1.2.x_wow.tpl.log 0m19.584s
(Full logs at [5])
# Description
All of these are on Mac OS X 10.7.3 on an SSD.
I'll be running the same set on spinning disk and then Robert N asked
me to populate the DBs not using builk docs. Since that's gonna take
a while, I'll probably run this overnight.
All of the results are generated by my fork of Jason's slow_couchdb[1]
and Filipe's seatoncouch[2].
The changes I've made is have the small_doc test run with 500k instead
of 50k docs, added .view files to match the tpl files in
seatoncouch/templates/* so we can have similar views use the different
doc structures.
I also added two scripts to orchestrate the above testing in a more
automated fashion. It also allows you to run the full matrix yourself.
All you need is set up homebrew allow `brew switch erlang R14B04` and
R15B (which is controlled in matrix.sh[3]) and have a git checkout of the
CouchDB sources that allow you to do `git checkout 1.1.1` or `1.2.x`
(which is controlled in runner.sh[4], adjust the path to the git checkout
there as well).
matrix.sh also allows you to specify which docs to run.
Please shout if you need any more info about this test run or how to
run this yourself.
# Analysis
Inconclusive, I'l like to run this on larger dbs in general to see if
there are more differences that shake out and I've yet have to run this
on a spinning disk let alone another OS* or more complex view functions
or larger design docs (like the one Stefan had).
* It shouldn't be too much work to port slow_couchdb to other OSs, I'll
definitely be looking into that, but we can do with every bit of help :)
So far, I'm happy to conclude that while there are definitely provable
differences, that we can live with them.
Cheers
Jan
--
[1]: https://github.com/janl/slow_couchdb
[2]: https://github.com/janl/seatoncouch
[3]: https://github.com/janl/slow_couchdb/blob/master/matrix.sh
[4]: https://github.com/janl/slow_couchdb/blob/master/runner.sh
[5]: http://jan.prima.de/slow_couch/ssd/
On Feb 28, 2012, at 18:53 , Filipe David Manana wrote:
> Jason, repeated my last test with the 1Kb docs (
> https://gist.github.com/1930804, map function
> http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
> as well. Here are the full results (also in
> https://gist.github.com/1930807):
>
>
> Before COUCHDB-1186
>
> fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":500000,"offset":0,"rows":[
> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
> ]}
>
> real 5m6.676s
> user 0m0.009s
> sys 0m0.010s
>
>
> After COUCHDB-1186
>
> fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":500000,"offset":0,"rows":[
> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
> ]}
>
> real 5m1.395s
> user 0m0.008s
> sys 0m0.010s
>
>
> After COUCHDB-1186 + better queueing patch
> (http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
>
> fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":500000,"offset":0,"rows":[
> {"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
> ]}
>
> real 4m48.175s
> user 0m0.008s
> sys 0m0.009s
>
>
> CouchDB branch 1.1.x
>
> fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":500000,"offset":0,"rows":[
> {"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
> ]}
>
> real 5m44.374s
> user 0m0.008s
> sys 0m0.010s
>
>
> Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
>
>
>
> On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=500000 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Filipe David Manana <fd...@apache.org>.
Jason, repeated my last test with the 1Kb docs (
https://gist.github.com/1930804, map function
http://friendpaste.com/5C99aqXocN6N6H1BAYIigs ) to cover branch 1.1.x
as well. Here are the full results (also in
https://gist.github.com/1930807):
Before COUCHDB-1186
fdmanana 23:21:05 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-a68a792-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-a68a792-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 5m6.676s
user 0m0.009s
sys 0m0.010s
After COUCHDB-1186
fdmanana 23:50:07 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 5m1.395s
user 0m0.008s
sys 0m0.010s
After COUCHDB-1186 + better queueing patch
(http://friendpaste.com/178nPFgfyyeGf2vtNRpL0w)
fdmanana 00:14:25 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0a-f023052-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0a-f023052-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"00051ef7-d735-48d7-9ba8-5a21a86e8d57","key":["dwarf","assassin"],"value":[{"x":31227.35,"y":31529.73},{"x":116667.85,"y":82008.25},{"x":224.11,"y":36652.41},{"x":128565.95,"y":6780.2},{"x":165230.43,"y":176208.63}]}
]}
real 4m48.175s
user 0m0.008s
sys 0m0.009s
CouchDB branch 1.1.x
fdmanana 08:16:58 ~/git/hub/slow_couchdb (master)> docs=500000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":500000,"offset":0,"rows":[
{"id":"0001c0a1-edcb-4dbc-aa9d-533c73d980cb","key":["dwarf","assassin"],"value":[{"x":62038.32,"y":105825.29},{"x":90713.13,"y":128570.97},{"x":43836.37,"y":80517.12},{"x":71610.97,"y":143739.99},{"x":86038.39,"y":84731.8}]}
]}
real 5m44.374s
user 0m0.008s
sys 0m0.010s
Disk model APPLE SSD TS128C, quad core machine, 8Gb of ram.
On Tue, Feb 28, 2012 at 5:17 AM, Jason Smith <jh...@apache.org> wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> docs=500000 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Dave Cottlehuber <da...@muse.net.nz>.
On 3 March 2012 00:56, Dave Cottlehuber <da...@muse.net.nz> wrote:
> # localhost, mac with vmware fusion internal network to windows 7 vm
> Disk: /dev/disk0
> Device / Media Name: APPLE SSD TS128C Media
> Solid State: Yes
> dave@akai slow_couchdb % ./bench.sh small_doc.tpl
With real disk and a low iMac. Some anomalies and differing builds
muddy the results:
http://friendpaste.com/6npWxkdhjqd0p3rHKBa3Kx
Have found Filipe's patch so will redo these tests tomorrow permitting.
A+
Dave
Re: Please report your indexing speed
Posted by Dave Cottlehuber <da...@muse.net.nz>.
# localhost, mac with vmware fusion internal network to windows 7 vm
Disk: /dev/disk0
Device / Media Name: APPLE SSD TS128C Media
Solid State: Yes
dave@akai slow_couchdb % ./bench.sh small_doc.tpl
Testing 50000 of small_doc.tpl in batches of 10000
all with js 1.8.5, I can compare 1.8.0 if interested.
Server: CouchDB/1.1.1 (Erlang OTP/R14B03)
real 0m13.800s
user 0m0.006s
sys 0m0.003s
Server: CouchDB/1.2.0 (Erlang OTP/R14B04)
real 0m19.185s
user 0m0.006s
sys 0m0.003s
Server: CouchDB/1.2.0 (Erlang OTP/R15B)
Date: Thu, 01 Mar 2012 12:50:47 GMT
real 0m18.428s
user 0m0.006s
sys 0m0.003s
Should get spinning disk (on a 2006 iMac) over the weekend with luck.
Is there a jira for this investigation?
A+
Dave
Re: Please report your indexing speed
Posted by Wendall Cada <we...@83864.com>.
Here are my 1.1.1 results on the same hardware/kernel/spidermonkey:
{"couchdb":"Welcome","version":"1.1.1"}
real 0m32.029s
user 0m0.000s
sys 0m0.006s
Wendall
On 02/29/2012 01:30 PM, Robert Newson wrote:
> Benoit, could you run the 1.7.0 test for longer? Say, double the size
> of BULK_COUNT?
>
> Wendall, that's a useful new data point. Any chance you could compare
> 1.1.1 with 1.2 so we can line it up with the other results?
>
> B.
>
> On 29 February 2012 19:40, Wendall Cada<we...@83864.com> wrote:
>> I'm using the bash script provided by Robert Newson.
>>
>> Linux version 3.2.7-1.fc16.i686.PAE
>> (mockbuild@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027 (Red
>> Hat 4.6.2-1) (GCC) )
>> spidermonkey 1.8.5
>> i686
>>
>> {"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
>> 2.fc16)
>> real 0m35.652s
>> user 0m0.001s
>> sys 0m0.006s
>>
>> {"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
>> real 0m20.134s
>> user 0m0.002s
>> sys 0m0.004s
>>
>> Wendall
>>
>>
>> On 02/28/2012 05:17 AM, Jason Smith wrote:
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
Benoit, could you run the 1.7.0 test for longer? Say, double the size
of BULK_COUNT?
Wendall, that's a useful new data point. Any chance you could compare
1.1.1 with 1.2 so we can line it up with the other results?
B.
On 29 February 2012 19:40, Wendall Cada <we...@83864.com> wrote:
> I'm using the bash script provided by Robert Newson.
>
> Linux version 3.2.7-1.fc16.i686.PAE
> (mockbuild@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027 (Red
> Hat 4.6.2-1) (GCC) )
> spidermonkey 1.8.5
> i686
>
> {"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
> 2.fc16)
> real 0m35.652s
> user 0m0.001s
> sys 0m0.006s
>
> {"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
> real 0m20.134s
> user 0m0.002s
> sys 0m0.004s
>
> Wendall
>
>
> On 02/28/2012 05:17 AM, Jason Smith wrote:
>>
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=500000 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
>
Re: Please report your indexing speed
Posted by Wendall Cada <we...@83864.com>.
I'm using the bash script provided by Robert Newson.
Linux version 3.2.7-1.fc16.i686.PAE
(mockbuild@x86-16.phx2.fedoraproject.org) (gcc version 4.6.2 20111027
(Red Hat 4.6.2-1) (GCC) )
spidermonkey 1.8.5
i686
{"couchdb":"Welcome","version":"1.0.3"} (from stock fedora rpm Release:
2.fc16)
real 0m35.652s
user 0m0.001s
sys 0m0.006s
{"couchdb":"Welcome","version":"1.2.0"} git 4cd60f3d1683
real 0m20.134s
user 0m0.002s
sys 0m0.004s
Wendall
On 02/28/2012 05:17 AM, Jason Smith wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> docs=500000 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Mar 4, 2012, at 18:40 , Jan Lehnardt wrote:
>
> On Mar 4, 2012, at 18:24 , Jan Lehnardt wrote:
>
>> I updated the google doc with results from an EC2 cc1.4xlarge instance (details are in the spreadsheet)
>>
>> This on EBS and Ubuntu 11.04/64.
>>
>> The results are bit different from the previous machine, but that isn't at all unexpected.
>>
>> tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference).
>
> Hah, I re-read through the results to make sure this is correct and I found a mistake. A copy and paste formula error accounted for bigger improvements of 1.2.x-filipe. This includes all my previous results.
>
> The good thing is 1.2.x-filipe is still faster, across the board than 1.1.1 and 1.2.x. Still significantly, but not *as* much as about 30% in all but one case.
>
> The tl;dr for the EC2 run can now be changed to that 1.2.x-filipe beats 1.1.1 and 1.2.x for all docs, it's just that for large docs (1000bytes), 1.2.x is faster than 1.1.1. But 1.2.x-filipe is even faster.
>
>
>> So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 for view builds.
Sorry for misquoting this line, it is new and the most significant of this email, I'll just repeat it :)
So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 for view builds.
--------------------------------------------------------------------------------------------
The bigger the docs, the better the results, on both SSD and spinning disk.
Cheers
Jan
--
>
>
> If you have any more hardware I could run this on, I'm happy to help with the setup, it isn't hard :)
>
> Cheers
> Jan
> --
>
>
>>
>> This still makes me want to include Filipe's patch into 1.2.x.
>>
>> Cheers
>> Jan
>> --
>>
>> On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote:
>>
>>> Hey all,
>>>
>>> I made another run with a bit of a different scenario.
>>>
>>>
>>> # The Scenario
>>>
>>> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>>>
>>> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>>>
>>> And a third matrix.sh (yay) that would run, on my system, different configurations.
>>>
>>> See https://gist.github.com/1971611 for the scripts.
>>>
>>> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>>>
>>> - Doc sizes 10, 100, 1000 bytes
>>> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
>>> - On an SSD and on a 5400rpm internal drive.
>>>
>>> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>>>
>>> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>>>
>>> # The Numbers
>>>
>>> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>>>
>>> See the "Grouped Data" sheet for comparisons.
>>>
>>> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>>>
>>>
>>> # Conclusion
>>>
>>> +1 to include Filipe's patch into 1.2.x.
>>>
>>>
>>>
>>> I'd love any feedback on methods, calculations and whatnot :)
>>>
>>> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>>>
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>>>
>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>
>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>
>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>> the test. For example:
>>>>
>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>
>>>> That should run the test and, God willing, upload the results to a
>>>> couch in the cloud. We should be able to use that information to
>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>> build, and how fast it ran. Modulo bugs.
>>>
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Mar 4, 2012, at 18:24 , Jan Lehnardt wrote:
> I updated the google doc with results from an EC2 cc1.4xlarge instance (details are in the spreadsheet)
>
> This on EBS and Ubuntu 11.04/64.
>
> The results are bit different from the previous machine, but that isn't at all unexpected.
>
> tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference).
Hah, I re-read through the results to make sure this is correct and I found a mistake. A copy and paste formula error accounted for bigger improvements of 1.2.x-filipe. This includes all my previous results.
The good thing is 1.2.x-filipe is still faster, across the board than 1.1.1 and 1.2.x. Still significantly, but not *as* much as about 30% in all but one case.
The tl;dr for the EC2 run can now be changed to that 1.2.x-filipe beats 1.1.1 and 1.2.x for all docs, it's just that for large docs (1000bytes), 1.2.x is faster than 1.1.1. But 1.2.x-filipe is even faster.
> So far, across the board, 1.2.x-filipe is ~16% faster (stdev 9%) than 1.1.1 for view builds.
If you have any more hardware I could run this on, I'm happy to help with the setup, it isn't hard :)
Cheers
Jan
--
>
> This still makes me want to include Filipe's patch into 1.2.x.
>
> Cheers
> Jan
> --
>
> On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote:
>
>> Hey all,
>>
>> I made another run with a bit of a different scenario.
>>
>>
>> # The Scenario
>>
>> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>>
>> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>>
>> And a third matrix.sh (yay) that would run, on my system, different configurations.
>>
>> See https://gist.github.com/1971611 for the scripts.
>>
>> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>>
>> - Doc sizes 10, 100, 1000 bytes
>> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
>> - On an SSD and on a 5400rpm internal drive.
>>
>> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>>
>> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>>
>> # The Numbers
>>
>> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>>
>> See the "Grouped Data" sheet for comparisons.
>>
>> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>>
>>
>> # Conclusion
>>
>> +1 to include Filipe's patch into 1.2.x.
>>
>>
>>
>> I'd love any feedback on methods, calculations and whatnot :)
>>
>> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>>
>>
>> Cheers
>> Jan
>> --
>>
>> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>>
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
I updated the google doc with results from an EC2 cc1.4xlarge instance (details are in the spreadsheet)
This on EBS and Ubuntu 11.04/64.
The results are bit different from the previous machine, but that isn't at all unexpected.
tl;dr: for small docs (10bytes, 100bytes) 1.2.x-filipe beats 1.2.x and 1.1.1 , for large docs (1000bytes), 1.2.x beats 1.2.x-filipe (6% difference).
This still makes me want to include Filipe's patch into 1.2.x.
Cheers
Jan
--
On Mar 4, 2012, at 10:24 , Jan Lehnardt wrote:
> Hey all,
>
> I made another run with a bit of a different scenario.
>
>
> # The Scenario
>
> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>
> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>
> And a third matrix.sh (yay) that would run, on my system, different configurations.
>
> See https://gist.github.com/1971611 for the scripts.
>
> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>
> - Doc sizes 10, 100, 1000 bytes
> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
> - On an SSD and on a 5400rpm internal drive.
>
> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>
> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>
> # The Numbers
>
> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>
> See the "Grouped Data" sheet for comparisons.
>
> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>
>
> # Conclusion
>
> +1 to include Filipe's patch into 1.2.x.
>
>
>
> I'd love any feedback on methods, calculations and whatnot :)
>
> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>
>
> Cheers
> Jan
> --
>
> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=500000 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Mar 4, 2012, at 13:03 , Bob Dionne wrote:
> Great Jan, so this confirms my back of the envelope test using Bob's script and Filipe's results. The patch is definitely helpful.
>
> I was wondering why no one had looked at test/bench, perhaps this more rigorous approach could provide the basis for a comprehensive performance tool
Good call!
I'd really like that our current efforts morph into a situation where we can `make perf` and get a bunch of good results to compare to other builds' `make perf`. Down the road, though, I think we need to write Erlang tools to do that, so Windows users can run them without too much trouble. (we could also bundle whatever scripting environment or C-based binaries with the builds, but since we already ship Erlang, we might as well use it :)
Cheers
Jan
--
>
> On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote:
>
>> Hey all,
>>
>> I made another run with a bit of a different scenario.
>>
>>
>> # The Scenario
>>
>> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>>
>> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>>
>> And a third matrix.sh (yay) that would run, on my system, different configurations.
>>
>> See https://gist.github.com/1971611 for the scripts.
>>
>> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>>
>> - Doc sizes 10, 100, 1000 bytes
>> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
>> - On an SSD and on a 5400rpm internal drive.
>>
>> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>>
>> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>>
>> # The Numbers
>>
>> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>>
>> See the "Grouped Data" sheet for comparisons.
>>
>> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>>
>>
>> # Conclusion
>>
>> +1 to include Filipe's patch into 1.2.x.
>>
>>
>>
>> I'd love any feedback on methods, calculations and whatnot :)
>>
>> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>>
>>
>> Cheers
>> Jan
>> --
>>
>> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>>
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
Excellent stuff, thanks Jan. I think it would be prudent to attempt to
identify which patch, or, more likely, which part of the patch, caused
the 30% regression and why. I will attempt that tomorrow or possibly
later today.
B.
On 4 March 2012 12:03, Bob Dionne <di...@dionne-associates.com> wrote:
> Great Jan, so this confirms my back of the envelope test using Bob's script and Filipe's results. The patch is definitely helpful.
>
> I was wondering why no one had looked at test/bench, perhaps this more rigorous approach could provide the basis for a comprehensive performance tool
>
> On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote:
>
>> Hey all,
>>
>> I made another run with a bit of a different scenario.
>>
>>
>> # The Scenario
>>
>> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>>
>> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>>
>> And a third matrix.sh (yay) that would run, on my system, different configurations.
>>
>> See https://gist.github.com/1971611 for the scripts.
>>
>> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>>
>> - Doc sizes 10, 100, 1000 bytes
>> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
>> - On an SSD and on a 5400rpm internal drive.
>>
>> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>>
>> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>>
>> # The Numbers
>>
>> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>>
>> See the "Grouped Data" sheet for comparisons.
>>
>> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>>
>>
>> # Conclusion
>>
>> +1 to include Filipe's patch into 1.2.x.
>>
>>
>>
>> I'd love any feedback on methods, calculations and whatnot :)
>>
>> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>>
>>
>> Cheers
>> Jan
>> --
>>
>> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>>
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>
Re: Please report your indexing speed
Posted by Bob Dionne <di...@dionne-associates.com>.
Great Jan, so this confirms my back of the envelope test using Bob's script and Filipe's results. The patch is definitely helpful.
I was wondering why no one had looked at test/bench, perhaps this more rigorous approach could provide the basis for a comprehensive performance tool
On Mar 4, 2012, at 4:24 AM, Jan Lehnardt wrote:
> Hey all,
>
> I made another run with a bit of a different scenario.
>
>
> # The Scenario
>
> I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
>
> I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
>
> And a third matrix.sh (yay) that would run, on my system, different configurations.
>
> See https://gist.github.com/1971611 for the scripts.
>
> I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
>
> - Doc sizes 10, 100, 1000 bytes
> - CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
> - On an SSD and on a 5400rpm internal drive.
>
> I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
>
> (The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
>
> # The Numbers
>
> See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
>
> See the "Grouped Data" sheet for comparisons.
>
> tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
>
>
> # Conclusion
>
> +1 to include Filipe's patch into 1.2.x.
>
>
>
> I'd love any feedback on methods, calculations and whatnot :)
>
> Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
>
>
> Cheers
> Jan
> --
>
> On Feb 28, 2012, at 14:17 , Jason Smith wrote:
>
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=500000 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
Hey all,
I made another run with a bit of a different scenario.
# The Scenario
I used a modified benchbulk.sh for inserting data (because it is an order of magnitude faster than the other methods we had). I added a command line parameter to specify the size of a single document in bytes (this was previously hardcoded in the script). Note that this script creates docs in a btree-friendly incrementing ID way.
I added a new script benchview.sh which is basically the lower part of Robert Newson's script. It creates a single view and queries it, measuring execution time of curl.
And a third matrix.sh (yay) that would run, on my system, different configurations.
See https://gist.github.com/1971611 for the scripts.
I ran ./benchbulk $size && ./benchview.sh for the following combinations, all on Mac OS X 10.7.3, Erlang R15B, Spidermonkey 1.8.5:
- Doc sizes 10, 100, 1000 bytes
- CouchDB 1.1.1, 1.2.x (as of last night), 1.2.x-filipe (as of last night + Filipe's patch from earlier in the thread)
- On an SSD and on a 5400rpm internal drive.
I ran each individual test three times and took the average to compare numbers. The full report (see below) includes each individual run's numbers)
(The gist includes the raw output data from matrix.sh for the 5400rpm run, for the SSDs, I don't have the original numbers anymore. I'm happy to re-run this, if you want that data as well.)
# The Numbers
See https://docs.google.com/spreadsheet/ccc?key=0AhESVUYnc_sQdDJ1Ry1KMTQ5enBDY0s1dHk2UVEzMHc for the full data set. It'd be great to get a second pair of eyes to make sure I didn't make any mistakes.
See the "Grouped Data" sheet for comparisons.
tl;dr: 1.2.x is about 30% slower and 1.2.x-filipe is about 30% faster than 1.1.1 in the scenario above.
# Conclusion
+1 to include Filipe's patch into 1.2.x.
I'd love any feedback on methods, calculations and whatnot :)
Also, I can run more variations, if you like, other Erlang or SpiderMokney versions e.g., just let me know.
Cheers
Jan
--
On Feb 28, 2012, at 14:17 , Jason Smith wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> docs=500000 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
go for it! Please mention "COUCHDB-1186" in the commit msg so we can trace it.
On 5 March 2012 10:17, Filipe David Manana <fd...@apache.org> wrote:
> On Mon, Mar 5, 2012 at 2:15 AM, Robert Newson <rn...@apache.org> wrote:
>> Filipe,
>>
>> Thanks for the explanation. I agree on the way forward and will apply
>> your patch to the relevant branches with attribution.
>
> Thanks. I'll do that myself soon unless someone in the meanwhile shows
> worse results with it.
>
>
>>
>> B.
>>
>> On 5 March 2012 07:41, Filipe David Manana <fd...@apache.org> wrote:
>>> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne <di...@dionne-associates.com> wrote:
>>>> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>>>>
>>>> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
>>>
>>> The explanation I gave in the 1.2.0 second round vote identifies the
>>> reason, which is that the updater is (depending on timings) collecting
>>> smaller batches of map results, which makes the btree updates less
>>> efficient (besides higher number of btree updates). The patch
>>> addresses this by queuing a batch of map results instead of queuing
>>> map results one by one. Jan's tests and mine are evidence that this is
>>> valid in practice and not just theory.
>>>
>>> The original main goal of COUCHDB-1186 was to make the indexing of
>>> views that emit reasonably large (or complex in structure) map values
>>> more efficient.
>>> Here's an example using Jason's slow_couchdb script with wow.tpl and
>>> map function of "function(doc) {emit([doc.type, doc.category],
>>> doc);}":
>>>
>>> 1.1.x:
>>>
>>> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=200000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":200000,"offset":0,"rows":[
>>> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
>>> ]}
>>>
>>> real 2m49.227s
>>> user 0m0.006s
>>> sys 0m0.005s
>>>
>>>
>>> 1.2.x:
>>>
>>> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=200000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":200000,"offset":0,"rows":[
>>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>>> ]}
>>>
>>> real 1m51.989s
>>> user 0m0.006s
>>> sys 0m0.004s
>>>
>>>
>>> 1.2.x + patch:
>>>
>>> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=200000
>>> batch=5000 ./bench.sh wow.tpl
>>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>>> {"couchdb":"Welcome","version":"1.2.0"}
>>>
>>> [INFO] Created DB named `db1'
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> (....)
>>> [INFO] Uploaded 5000 documents via _bulk_docs
>>> Building view.
>>> {"total_rows":200000,"offset":0,"rows":[
>>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>>> ]}
>>>
>>> real 1m45.573s
>>> user 0m0.006s
>>> sys 0m0.004s
>>>
>>>
>>> Unless someone comes up with scenarios where 1.2.x with the patch is
>>> significantly slower than 1.1.x, I think we should close this and move
>>> to release 1.2.0.
>>>
>>> Thanks all for the testing.
>>>
>>>>
>>>> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>>>>
>>>>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>>>>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>>>>> other thread and it brings the performance back to the 1.1.x level.
>>>>>
>>>>> I'd say that patch is a +1
>>>>>
>>>>> 1.2.x
>>>>> real 3m3.093s
>>>>> user 0m0.028s
>>>>> sys 0m0.008s
>>>>> ==================
>>>>> 1.1.x
>>>>> real 2m16.609s
>>>>> user 0m0.026s
>>>>> sys 0m0.007s
>>>>> =================
>>>>> 1.2.x with patch to couch_os_process reverted
>>>>> real 3m7.012s
>>>>> user 0m0.029s
>>>>> sys 0m0.008s
>>>>> =================
>>>>> 1.2.x with Filipe's katest patch to couch_view_updater
>>>>> real 2m11.038s
>>>>> user 0m0.028s
>>>>> sys 0m0.007s
>>>>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>>>>
>>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>>
>>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>>
>>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>>> the test. For example:
>>>>>>
>>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>>
>>>>>> That should run the test and, God willing, upload the results to a
>>>>>> couch in the cloud. We should be able to use that information to
>>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>>> build, and how fast it ran. Modulo bugs.
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Filipe David Manana,
>>>
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Filipe David Manana <fd...@apache.org>.
On Mon, Mar 5, 2012 at 2:15 AM, Robert Newson <rn...@apache.org> wrote:
> Filipe,
>
> Thanks for the explanation. I agree on the way forward and will apply
> your patch to the relevant branches with attribution.
Thanks. I'll do that myself soon unless someone in the meanwhile shows
worse results with it.
>
> B.
>
> On 5 March 2012 07:41, Filipe David Manana <fd...@apache.org> wrote:
>> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne <di...@dionne-associates.com> wrote:
>>> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>>>
>>> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
>>
>> The explanation I gave in the 1.2.0 second round vote identifies the
>> reason, which is that the updater is (depending on timings) collecting
>> smaller batches of map results, which makes the btree updates less
>> efficient (besides higher number of btree updates). The patch
>> addresses this by queuing a batch of map results instead of queuing
>> map results one by one. Jan's tests and mine are evidence that this is
>> valid in practice and not just theory.
>>
>> The original main goal of COUCHDB-1186 was to make the indexing of
>> views that emit reasonably large (or complex in structure) map values
>> more efficient.
>> Here's an example using Jason's slow_couchdb script with wow.tpl and
>> map function of "function(doc) {emit([doc.type, doc.category],
>> doc);}":
>>
>> 1.1.x:
>>
>> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=200000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":200000,"offset":0,"rows":[
>> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
>> ]}
>>
>> real 2m49.227s
>> user 0m0.006s
>> sys 0m0.005s
>>
>>
>> 1.2.x:
>>
>> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=200000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":200000,"offset":0,"rows":[
>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>> ]}
>>
>> real 1m51.989s
>> user 0m0.006s
>> sys 0m0.004s
>>
>>
>> 1.2.x + patch:
>>
>> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=200000
>> batch=5000 ./bench.sh wow.tpl
>> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
>> {"couchdb":"Welcome","version":"1.2.0"}
>>
>> [INFO] Created DB named `db1'
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> (....)
>> [INFO] Uploaded 5000 documents via _bulk_docs
>> Building view.
>> {"total_rows":200000,"offset":0,"rows":[
>> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
>> ]}
>>
>> real 1m45.573s
>> user 0m0.006s
>> sys 0m0.004s
>>
>>
>> Unless someone comes up with scenarios where 1.2.x with the patch is
>> significantly slower than 1.1.x, I think we should close this and move
>> to release 1.2.0.
>>
>> Thanks all for the testing.
>>
>>>
>>> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>>>
>>>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>>>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>>>> other thread and it brings the performance back to the 1.1.x level.
>>>>
>>>> I'd say that patch is a +1
>>>>
>>>> 1.2.x
>>>> real 3m3.093s
>>>> user 0m0.028s
>>>> sys 0m0.008s
>>>> ==================
>>>> 1.1.x
>>>> real 2m16.609s
>>>> user 0m0.026s
>>>> sys 0m0.007s
>>>> =================
>>>> 1.2.x with patch to couch_os_process reverted
>>>> real 3m7.012s
>>>> user 0m0.029s
>>>> sys 0m0.008s
>>>> =================
>>>> 1.2.x with Filipe's katest patch to couch_view_updater
>>>> real 2m11.038s
>>>> user 0m0.028s
>>>> sys 0m0.007s
>>>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>>>
>>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>>
>>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>>
>>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>>> the test. For example:
>>>>>
>>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>>
>>>>> That should run the test and, God willing, upload the results to a
>>>>> couch in the cloud. We should be able to use that information to
>>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>>> build, and how fast it ran. Modulo bugs.
>>>>
>>>
>>
>>
>>
>> --
>> Filipe David Manana,
>>
>> "Reasonable men adapt themselves to the world.
>> Unreasonable men adapt the world to themselves.
>> That's why all progress depends on unreasonable men."
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Robert Newson <rn...@apache.org>.
Filipe,
Thanks for the explanation. I agree on the way forward and will apply
your patch to the relevant branches with attribution.
B.
On 5 March 2012 07:41, Filipe David Manana <fd...@apache.org> wrote:
> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne <di...@dionne-associates.com> wrote:
>> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>>
>> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
>
> The explanation I gave in the 1.2.0 second round vote identifies the
> reason, which is that the updater is (depending on timings) collecting
> smaller batches of map results, which makes the btree updates less
> efficient (besides higher number of btree updates). The patch
> addresses this by queuing a batch of map results instead of queuing
> map results one by one. Jan's tests and mine are evidence that this is
> valid in practice and not just theory.
>
> The original main goal of COUCHDB-1186 was to make the indexing of
> views that emit reasonably large (or complex in structure) map values
> more efficient.
> Here's an example using Jason's slow_couchdb script with wow.tpl and
> map function of "function(doc) {emit([doc.type, doc.category],
> doc);}":
>
> 1.1.x:
>
> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
> ]}
>
> real 2m49.227s
> user 0m0.006s
> sys 0m0.005s
>
>
> 1.2.x:
>
> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m51.989s
> user 0m0.006s
> sys 0m0.004s
>
>
> 1.2.x + patch:
>
> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m45.573s
> user 0m0.006s
> sys 0m0.004s
>
>
> Unless someone comes up with scenarios where 1.2.x with the patch is
> significantly slower than 1.1.x, I think we should close this and move
> to release 1.2.0.
>
> Thanks all for the testing.
>
>>
>> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>>
>>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>>> other thread and it brings the performance back to the 1.1.x level.
>>>
>>> I'd say that patch is a +1
>>>
>>> 1.2.x
>>> real 3m3.093s
>>> user 0m0.028s
>>> sys 0m0.008s
>>> ==================
>>> 1.1.x
>>> real 2m16.609s
>>> user 0m0.026s
>>> sys 0m0.007s
>>> =================
>>> 1.2.x with patch to couch_os_process reverted
>>> real 3m7.012s
>>> user 0m0.029s
>>> sys 0m0.008s
>>> =================
>>> 1.2.x with Filipe's katest patch to couch_view_updater
>>> real 2m11.038s
>>> user 0m0.028s
>>> sys 0m0.007s
>>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>>
>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>
>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>
>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>> the test. For example:
>>>>
>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>
>>>> That should run the test and, God willing, upload the results to a
>>>> couch in the cloud. We should be able to use that information to
>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>> build, and how fast it ran. Modulo bugs.
>>>
>>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Wendall Cada <we...@83864.com>.
My first tests for Centos were against erlang R12B, I'm seeing yet
another speed improvement using R14B04 on Centos 6.
{"couchdb":"Welcome","version":"1.2.0"}
real 4m45.824s
user 0m0.003s
sys 0m0.003s
Huge improvement over my current production scenario R12B, Spidermonkey
1.7, Couchdb 1.0.2, index rebuilding takes 13m16.480s
Great job on the speed improvments!
Wendall
On 03/06/2012 01:19 PM, Wendall Cada wrote:
> Here is another set of results using a db with ~300k docs and ~10
> views in the design doc.
>
> Fedora Core2Duo 2GHz, 7200RPM disk:
> {"couchdb":"Welcome","version":"1.0.3"} (spidermonkey 1.8.5) (stock
> package)
> real 9m9.086s
> user 0m0.007s
> sys 0m0.016s
>
> {"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
> real 7m59.437s
> user 0m0.007s
> sys 0m0.016s
>
> {"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
> (git-db8335bf2a)
>
> real 6m25.469s
> user 0m0.009s
> sys 0m0.027s
>
> Centos Xeon 2.27GHz (2 cores) 15k SCSI/RAID 10:
> {"couchdb":"Welcome","version":"1.0.1"} (spidermonkey 1.7.0) (stock
> package)
> real 13m16.480s
> user 0m0.001s
> sys 0m0.004s
>
> {"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
> real 6m14.999s
> user 0m0.001s
> sys 0m0.003s
>
> {"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
> (git-fb72251bc7)
> real 5m24.925s
> user 0m0.002s
> sys 0m0.004s
>
> While there are certainly speed improvements over 1.1.1, the biggest
> change is moving to a newer spidermonkey.
>
> Wendall
Re: Please report your indexing speed
Posted by Wendall Cada <we...@83864.com>.
Here is another set of results using a db with ~300k docs and ~10 views
in the design doc.
Fedora Core2Duo 2GHz, 7200RPM disk:
{"couchdb":"Welcome","version":"1.0.3"} (spidermonkey 1.8.5) (stock
package)
real 9m9.086s
user 0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real 7m59.437s
user 0m0.007s
sys 0m0.016s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-db8335bf2a)
real 6m25.469s
user 0m0.009s
sys 0m0.027s
Centos Xeon 2.27GHz (2 cores) 15k SCSI/RAID 10:
{"couchdb":"Welcome","version":"1.0.1"} (spidermonkey 1.7.0) (stock package)
real 13m16.480s
user 0m0.001s
sys 0m0.004s
{"couchdb":"Welcome","version":"1.1.1"} (spidermonkey 1.8.5) (git-1.1.1)
real 6m14.999s
user 0m0.001s
sys 0m0.003s
{"couchdb":"Welcome","version":"1.2.0"} (spidermonkey 1.8.5)
(git-fb72251bc7)
real 5m24.925s
user 0m0.002s
sys 0m0.004s
While there are certainly speed improvements over 1.1.1, the biggest
change is moving to a newer spidermonkey.
Wendall
Re: Please report your indexing speed
Posted by Bob Dionne <di...@dionne-associates.com>.
Awesome Filipe, so these two were related, I didn't get that subtlety in your original post. This is great, thanks for the patch -- Bob
On Mar 5, 2012, at 2:41 AM, Filipe David Manana wrote:
> On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne <di...@dionne-associates.com> wrote:
>> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>>
>> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
>
> The explanation I gave in the 1.2.0 second round vote identifies the
> reason, which is that the updater is (depending on timings) collecting
> smaller batches of map results, which makes the btree updates less
> efficient (besides higher number of btree updates). The patch
> addresses this by queuing a batch of map results instead of queuing
> map results one by one. Jan's tests and mine are evidence that this is
> valid in practice and not just theory.
>
> The original main goal of COUCHDB-1186 was to make the indexing of
> views that emit reasonably large (or complex in structure) map values
> more efficient.
> Here's an example using Jason's slow_couchdb script with wow.tpl and
> map function of "function(doc) {emit([doc.type, doc.category],
> doc);}":
>
> 1.1.x:
>
> fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
> ]}
>
> real 2m49.227s
> user 0m0.006s
> sys 0m0.005s
>
>
> 1.2.x:
>
> fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m51.989s
> user 0m0.006s
> sys 0m0.004s
>
>
> 1.2.x + patch:
>
> fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=200000
> batch=5000 ./bench.sh wow.tpl
> Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
> {"couchdb":"Welcome","version":"1.2.0"}
>
> [INFO] Created DB named `db1'
> [INFO] Uploaded 5000 documents via _bulk_docs
> (....)
> [INFO] Uploaded 5000 documents via _bulk_docs
> Building view.
> {"total_rows":200000,"offset":0,"rows":[
> {"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
> ]}
>
> real 1m45.573s
> user 0m0.006s
> sys 0m0.004s
>
>
> Unless someone comes up with scenarios where 1.2.x with the patch is
> significantly slower than 1.1.x, I think we should close this and move
> to release 1.2.0.
>
> Thanks all for the testing.
>
>>
>> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>>
>>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>>> other thread and it brings the performance back to the 1.1.x level.
>>>
>>> I'd say that patch is a +1
>>>
>>> 1.2.x
>>> real 3m3.093s
>>> user 0m0.028s
>>> sys 0m0.008s
>>> ==================
>>> 1.1.x
>>> real 2m16.609s
>>> user 0m0.026s
>>> sys 0m0.007s
>>> =================
>>> 1.2.x with patch to couch_os_process reverted
>>> real 3m7.012s
>>> user 0m0.029s
>>> sys 0m0.008s
>>> =================
>>> 1.2.x with Filipe's katest patch to couch_view_updater
>>> real 2m11.038s
>>> user 0m0.028s
>>> sys 0m0.007s
>>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>>
>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>
>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>
>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>> the test. For example:
>>>>
>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>
>>>> That should run the test and, God willing, upload the results to a
>>>> couch in the cloud. We should be able to use that information to
>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>> build, and how fast it ran. Modulo bugs.
>>>
>>
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Filipe David Manana <fd...@apache.org>.
On Sun, Mar 4, 2012 at 9:45 AM, Bob Dionne <di...@dionne-associates.com> wrote:
> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>
> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
The explanation I gave in the 1.2.0 second round vote identifies the
reason, which is that the updater is (depending on timings) collecting
smaller batches of map results, which makes the btree updates less
efficient (besides higher number of btree updates). The patch
addresses this by queuing a batch of map results instead of queuing
map results one by one. Jan's tests and mine are evidence that this is
valid in practice and not just theory.
The original main goal of COUCHDB-1186 was to make the indexing of
views that emit reasonably large (or complex in structure) map values
more efficient.
Here's an example using Jason's slow_couchdb script with wow.tpl and
map function of "function(doc) {emit([doc.type, doc.category],
doc);}":
1.1.x:
fdmanana 07:04:12 ~/git/hub/slow_couchdb (master)> docs=200000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.1.2a785d32f-git (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.1.2a785d32f-git"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":200000,"offset":0,"rows":[
{"id":"00144af5-9f07-448e-af88-026674e3e3d0","key":["dwarf","assassin"],"value":{"_id":"00144af5-9f07-448e-af88-026674e3e3d0","_rev":"1-785fbf5e641f3d10fa083501ad82a9fe","data3":"Vl6BftQEWY6imvNs0FasOj2CrPCptP70z5d","ratio":1.6,"integers":[48028,3170,54066,95547,70643,23763,25804,33180,89061,35274,48244,91792,37936,11855],"category":"assassin","nested":{"dict":{"3XGVdTTF":31490,"SDxKa54e":40,"XIzUloRH":7,"5Mj9F4bp":192,"1sXfjgYf":1203,"XP5YSqhX":25461,"QJr0Xhxn":9941},"string1":"3Q4tvmhHwKvedKiRnoL6xUz","string2":"dWI1mrrAypRh","values":[33712,57371,88567,88361,70873,6327,17326,91004,41840,86257],"string3":"i7OGysnXvynz41VMQJ","coords":[{"x":65350.46,"y":103881.18},{"x":24180.14,"y":8474.9},{"x":88326.66,"y":43151.76},{"x":120199.77,"y":102270.29},{"x":191924.18,"y":74479.75}]},"level":21,"type":"dwarf","data1":"Vpkplo80LshlcjBE0ySJNNpfgDy2bu8byWrmZ44B","data2":"GnyNbos75Wxm1C5MLdOeXNniHamBjld70vHqoJnEtnlfekuPXJ"}}
]}
real 2m49.227s
user 0m0.006s
sys 0m0.005s
1.2.x:
fdmanana 07:13:30 ~/git/hub/slow_couchdb (master)> docs=200000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":200000,"offset":0,"rows":[
{"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
]}
real 1m51.989s
user 0m0.006s
sys 0m0.004s
1.2.x + patch:
fdmanana 07:29:11 ~/git/hub/slow_couchdb (master)> docs=200000
batch=5000 ./bench.sh wow.tpl
Server: CouchDB/1.2.0 (Erlang OTP/R14B03)
{"couchdb":"Welcome","version":"1.2.0"}
[INFO] Created DB named `db1'
[INFO] Uploaded 5000 documents via _bulk_docs
(....)
[INFO] Uploaded 5000 documents via _bulk_docs
Building view.
{"total_rows":200000,"offset":0,"rows":[
{"id":"0005cd07-49f4-4a99-b506-acef948f2acc","key":["dwarf","assassin"],"value":{"_id":"0005cd07-49f4-4a99-b506-acef948f2acc","_rev":"1-4b418e69618bf11124a03e1a3845f071","data3":"T0W2JBUET9yzRXHfUqcUBwFhYGKh14YFVxk","ratio":1.6999999999999999556,"integers":[25658,7573,47779,43217,49586,57992,13549,90984,45253,49560,1643,64085,38381,62544],"category":"assassin","nested":{"dict":{"oWhW4jJ6":199,"EPSVtKtS":5638,"8WpzvD5x":73714,"stD9Ynfh":8924,"0qh5Nc1g":5994,"pBa5vJyy":18,"s25oAkRc":165270},"string1":"fNNHb8lxtcy7GpwSU3yRyaC","string2":"rilbiZM7yAaK","values":[49632,93665,73258,75675,4229,15742,16965,76825,22049,79829],"string3":"IwX09SiOLMSSyxffMB","coords":[{"x":179620.45000000001164,"y":11539.989999999999782},{"x":68483.820000000006985,"y":110559.19999999999709},{"x":67197.940000000002328,"y":96702.210000000006403},{"x":25469.869999999998981,"y":79049.490000000005239},{"x":157059.89999999999418,"y":34963.410000000003492}]},"level":6,"type":"dwarf","data1":"njpz38JSfz00p2Lc2Jv0dON7UfTljRgz0J2B7w7K","data2":"4hpsT2szDrssbUCHEirTzHOIhSxTd83i1FO5aNXgoGAfO2srH1"}}
]}
real 1m45.573s
user 0m0.006s
sys 0m0.004s
Unless someone comes up with scenarios where 1.2.x with the patch is
significantly slower than 1.1.x, I think we should close this and move
to release 1.2.0.
Thanks all for the testing.
>
> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>
>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>> other thread and it brings the performance back to the 1.1.x level.
>>
>> I'd say that patch is a +1
>>
>> 1.2.x
>> real 3m3.093s
>> user 0m0.028s
>> sys 0m0.008s
>> ==================
>> 1.1.x
>> real 2m16.609s
>> user 0m0.026s
>> sys 0m0.007s
>> =================
>> 1.2.x with patch to couch_os_process reverted
>> real 3m7.012s
>> user 0m0.029s
>> sys 0m0.008s
>> =================
>> 1.2.x with Filipe's katest patch to couch_view_updater
>> real 2m11.038s
>> user 0m0.028s
>> sys 0m0.007s
>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>
--
Filipe David Manana,
"Reasonable men adapt themselves to the world.
Unreasonable men adapt the world to themselves.
That's why all progress depends on unreasonable men."
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Mar 4, 2012, at 18:48 , Jan Lehnardt wrote:
>
> On Mar 4, 2012, at 18:45 , Bob Dionne wrote:
>
>> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
>
> I see ~10% faster than where 1.1.1 was for small docs and ~30% for large docs.
I got curious whether there was a drop-off (or up, I guess) point for the performance improvement between doc size and %-improved.
I re-ran the tests for 300, 500 & 700 byte docs respectively and the numbers (updated in the spreadsheet) suggest a ~5% improvement on each iteration.
So, looks linear, no sweet spot.
Moving on.
Cheers
Jan
--
>
>
>> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
>
> +1
>
> Cheers
> Jan
> --
>
>
>
>>
>> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>>
>>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>>> other thread and it brings the performance back to the 1.1.x level.
>>>
>>> I'd say that patch is a +1
>>>
>>> 1.2.x
>>> real 3m3.093s
>>> user 0m0.028s
>>> sys 0m0.008s
>>> ==================
>>> 1.1.x
>>> real 2m16.609s
>>> user 0m0.026s
>>> sys 0m0.007s
>>> =================
>>> 1.2.x with patch to couch_os_process reverted
>>> real 3m7.012s
>>> user 0m0.029s
>>> sys 0m0.008s
>>> =================
>>> 1.2.x with Filipe's katest patch to couch_view_updater
>>> real 2m11.038s
>>> user 0m0.028s
>>> sys 0m0.007s
>>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>>
>>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>>
>>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>>
>>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>>> the test. For example:
>>>>
>>>> docs=500000 ./bench.sh small_doc.tpl
>>>>
>>>> That should run the test and, God willing, upload the results to a
>>>> couch in the cloud. We should be able to use that information to
>>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>>> build, and how fast it ran. Modulo bugs.
>>>
>>
>
Re: Please report your indexing speed
Posted by Jan Lehnardt <ja...@apache.org>.
On Mar 4, 2012, at 18:45 , Bob Dionne wrote:
> yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
I see ~10% faster than where 1.1.1 was for small docs and ~30% for large docs.
> I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
+1
Cheers
Jan
--
>
> On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
>
>> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
>> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
>> other thread and it brings the performance back to the 1.1.x level.
>>
>> I'd say that patch is a +1
>>
>> 1.2.x
>> real 3m3.093s
>> user 0m0.028s
>> sys 0m0.008s
>> ==================
>> 1.1.x
>> real 2m16.609s
>> user 0m0.026s
>> sys 0m0.007s
>> =================
>> 1.2.x with patch to couch_os_process reverted
>> real 3m7.012s
>> user 0m0.029s
>> sys 0m0.008s
>> =================
>> 1.2.x with Filipe's katest patch to couch_view_updater
>> real 2m11.038s
>> user 0m0.028s
>> sys 0m0.007s
>> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>>
>>> Forgive the clean new thread. Hopefully it will not remain so.
>>>
>>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>>
>>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>>> the test. For example:
>>>
>>> docs=500000 ./bench.sh small_doc.tpl
>>>
>>> That should run the test and, God willing, upload the results to a
>>> couch in the cloud. We should be able to use that information to
>>> identify who you are, whether you are on SSD, what Erlang and Couch
>>> build, and how fast it ran. Modulo bugs.
>>
>
Re: Please report your indexing speed
Posted by Bob Dionne <di...@dionne-associates.com>.
yes, I was surprised by the 30% claim as my numbers showed it only getting back to where we were with 1.1.x
I think Bob's suggestion to get to the root code change that caused this regression is important as it will help us assess all the other cases this testing hasn't even touched yet
On Mar 3, 2012, at 5:25 PM, Bob Dionne wrote:
> I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
> to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
> other thread and it brings the performance back to the 1.1.x level.
>
> I'd say that patch is a +1
>
> 1.2.x
> real 3m3.093s
> user 0m0.028s
> sys 0m0.008s
> ==================
> 1.1.x
> real 2m16.609s
> user 0m0.026s
> sys 0m0.007s
> =================
> 1.2.x with patch to couch_os_process reverted
> real 3m7.012s
> user 0m0.029s
> sys 0m0.008s
> =================
> 1.2.x with Filipe's katest patch to couch_view_updater
> real 2m11.038s
> user 0m0.028s
> sys 0m0.007s
> On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
>
>> Forgive the clean new thread. Hopefully it will not remain so.
>>
>> If you can, would you please clone https://github.com/jhs/slow_couchdb
>>
>> And build whatever Erlangs and CouchDB checkouts you see fit, and run
>> the test. For example:
>>
>> docs=500000 ./bench.sh small_doc.tpl
>>
>> That should run the test and, God willing, upload the results to a
>> couch in the cloud. We should be able to use that information to
>> identify who you are, whether you are on SSD, what Erlang and Couch
>> build, and how fast it ran. Modulo bugs.
>
Re: Please report your indexing speed
Posted by Bob Dionne <di...@dionne-associates.com>.
I ran some tests, using Bob's latest script. The first versus the second clearly show the regression. The third is the 1.2.x with the patch
to couch_os_process reverted and it seems to have no impact. The last has Filipe's latest patch to couch_view_updater discussed in the
other thread and it brings the performance back to the 1.1.x level.
I'd say that patch is a +1
1.2.x
real 3m3.093s
user 0m0.028s
sys 0m0.008s
==================
1.1.x
real 2m16.609s
user 0m0.026s
sys 0m0.007s
=================
1.2.x with patch to couch_os_process reverted
real 3m7.012s
user 0m0.029s
sys 0m0.008s
=================
1.2.x with Filipe's katest patch to couch_view_updater
real 2m11.038s
user 0m0.028s
sys 0m0.007s
On Feb 28, 2012, at 8:17 AM, Jason Smith wrote:
> Forgive the clean new thread. Hopefully it will not remain so.
>
> If you can, would you please clone https://github.com/jhs/slow_couchdb
>
> And build whatever Erlangs and CouchDB checkouts you see fit, and run
> the test. For example:
>
> docs=500000 ./bench.sh small_doc.tpl
>
> That should run the test and, God willing, upload the results to a
> couch in the cloud. We should be able to use that information to
> identify who you are, whether you are on SSD, what Erlang and Couch
> build, and how fast it ran. Modulo bugs.