You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Mike Coolin <mc...@techie.com> on 2012/03/30 06:36:59 UTC
couchdb-stress testing
Hello all,
I've started to put together some tools to test couchdb in a number of ways. Ideally this tool will provide a means to new users to better understand how couch will perform in their environments, additionally it could be used to assist then in tuning their system for expected loads.
When I first threw this idea out there I got a number of suggestion. One big time saver was to use a proven erlang tool for testing, so I have been looking into it. tsung (http://tsung.erlang-projects.org/ ), is a testing tool that can scale, and its written in erlang. It's really not to hard to get running, I'd recommend installing from the site and running though the build process. Some of the packed versions out there are very old and don't work. The documentation is weak. That said I did an exercise to get the xml required to run the tests. the results were much bigger than expected but futon does not just exercise the code but often requests object from the web server creating allot of background noise. That not really a problem as tsung does provide a nice little recorder for capturing the xml that needs to be written and now it just a matter of whipping it into the appropriate loops.
The project can be found at: git://github.com/mcoolin/couchdb-stress.git
Don't go running off their yet there's not much to see but some analysis documents and some rough ideas I've thrown together. Please email or add to the project if you can.
I invite you to toss you ideas and suggestion for testing and testing profiles/approaches that you think would be helpful.
I fully expect that there will be several testing files that treat the server in different ways, such as a read only vs logging vs read/write. multiple databases and multiple users on each. Tsung can be run on the same machine as couch, but it can also be placed on a server farm and produce millions of requests for stated periods of time.
It is not my intention to repeat the futon test in tsung, I just used if to scrape of the xml I'll need for writing tests. But the exercise did reveal a few surprises. Some 7 sections failed, likely due to some special handling needed by tsung or maybe some feature in couchdb or timing issues.
Testing was done against today's 1.2 version, please note that the trailing number refers to a page number of the results. I captured the logs of the errors and the calls that were made for those of you who are inclined to investigate. In the next few days I will try to get a sample going of at least one type of test. I'm pretty green with couchdb and tsung so if you have suggestions I'm all ears, I can create specific test suites that reveal warts with your help that hopefully in future will help those developing make couch even better.
Testing recording of futon 1.2.0 with tsung-recorder 1
Basics - failed 6
XML 6
Logs 9
all_docs - passed 10
XML 10
Logs – NONE 16
attachments - passed 16
XML 16
Logs - None 31
31
attachments_multipart - passed 31
XML 31
Logs - None 45
45
attachment_names - Passed 45
XML 45
Logs - none 47
47
attachment_paths - passed 47
XML 47
Logs - none 51
attachment_ranges - passed 51
XML 51
Logs - none 52
attachment_views - passed 52
XML 52
Logs - none 57
57
auth_cache -passed 57
XML 57
Logs - none 63
63
batch_save - passed 63
XML 63
Logs - none 83
83
bulk_docs- passed 83
XML 83
Logs - none 86
86
Changes - passed 86
XML 86
Logs – none 94
Coffee - Passed 94
XML 94
Logs - none 96
Compact - failed – eventually passed timing? 96
XML 96
Logs 100
config - passed 120
XML 120
Logs - none 123
123
Conflicts - passed 123
XML 123
Logs - none 126
126
content_negotiation - passed 126
XML 126
Logs - none 127
127
cookie_auth - passed 127
XML 127
Logs - none 133
133
copy_doc - passed 133
XML 133
Logs - none 135
135
delayed_commits - failed 135
XML 135
Logs 140
391
design_docs - failed 391
XML 391
Logs 418
design_options - passed 457
XML 457
Logs - none 458
458
design_paths - passed 458
XML 458
Logs - none 460
460
erlang_views - passed 460
XML 460
Logs - none 556
556
etags_head - passed 556
XML 556
Logs- none 557
557
etags_views-failed 557
XML 557
Logs 564
658
form_submit - passed 658
XML 658
Logs - none 659
659
Http - passed 659
XML 659
Logs - none 660
660
invalid_docids - passed 660
XML 660
Logs - none 662
662
Jsonp - passed 662
XML 662
Logs - none 663
663
large_docs - passed 663
XML 663
Logs - none 676
676
list_views - passed 676
XML 676
Logs - none 681
681
lots_of_docs - passed 681
XML 681
Logs - none 686
686
method_override - passed 686
XML 686
Logs - none 687
687
multiple_rows - passed 687
XML 687
Logs - none 689
689
Oauth - passed 689
XML 689
Logs - none 698
Oauth_users_db - passed 698
XML 698
Logs - none 701
Proxyauth - passed 701
XML 701
Logs - none 704
704
Purge - passed 704
XML 704
Logs - none 708
708
reader_acl - failed 708
XML 708
Logs 713
797
recreate_doc - passed 797
XML 797
Logs - none 803
803
Reduce - passed 803
XML 803
Logs - none 809
809
reduce_builtin - passed 809
XML 809
Logs - none 866
866
reduce_false - passed 866
XML 866
Logs - none 868
868
reduce_false_temp - passed 868
XML 868
Logs - none 870
870
Replication - passed 870
XML 870
Logs - none 871
replicator_db - failed 871
XML 871
Logs 878
replicator_db_security - passed 970
XML 970
Logs - none 978
rev_stemming - passed 978
XML 978
Logs - none 982
982
Rewrite - passed 982
XML 982
Logs - none 988
988
security_validation - passed 988
XML 988
Logs - none 1000
1000
show_documents - passed 1000
XML 1000
Logs - none 1008
1008
Stats - fails 1008
XML 1008
Logs 1009
1021
update_documents - passed 1021
XML 1021
Logs - none 1024
1024
users_db - passed 1024
XML 1024
Logs - none 1027
users_db_security - passed 1027
XML 1027
Logs - none 1032
1032
Utf8 - passed 1032
XML 1032
Logs - none 1033
1033
Uuids - passed 1033
XML 1033
Logs - none 1035
1035
view_collation - passed 1035
XML 1035
Logs - none 1041
1041
view_collation_raw - passed 1041
XML 1041
Logs - none 1046
1046
view_conflicts - passed 1046
XML 1046
Logs - none 1047
1047
view_compaction - passed 1047
XML 1047
Logs - none 1171
1171
view_errors - passed 1171
XML 1171
Logs - none 1174
1174
view_include_docs - passed 1174
XML 1174
Logs - none 1179
1179
view_multi_key_all_docs - passed 1179
XML 1179
Logs - none 1184
1184
view_multi_key_design - passed 1184
XML 1184
Logs - none 1188
1188
view_multi_key_temp - passed 1188
XML 1188
Logs - none 1194
1194
view_offsets - passed 1194
XML 1194
Logs- none 1209
1209
view_pagination - passed 1209
XML 1209
Logs - none 1217
1217
view_sandboxing - passed 1217
XML 1217
Logs - none 1220
1220
view_update_seq - passed 1220
XML 1220
Logs - none 1228
1228
view_xml - passed 1228
XML 1228
Logs - passed 1230
Re: couchdb-stress testing
Posted by Dave Cottlehuber <da...@muse.net.nz>.
On 30 March 2012 06:36, Mike Coolin <mc...@techie.com> wrote:
> Hello all,
>
> I've started to put together some tools to test couchdb in a number of ways. Ideally this tool will provide a means to new users to better understand how couch will perform in their environments, additionally it could be used to assist then in tuning their system for expected loads.
>
> When I first threw this idea out there I got a number of suggestion. One big time saver was to use a proven erlang tool for testing, so I have been looking into it. tsung (http://tsung.erlang-projects.org/ ), is a testing tool that can scale, and its written in erlang. It's really not to hard to get running, I'd recommend installing from the site and running though the build process. Some of the packed versions out there are very old and don't work. The documentation is weak. That said I did an exercise to get the xml required to run the tests. the results were much bigger than expected but futon does not just exercise the code but often requests object from the web server creating allot of background noise. That not really a problem as tsung does provide a nice little recorder for capturing the xml that needs to be written and now it just a matter of whipping it into the appropriate loops.
>
> The project can be found at: git://github.com/mcoolin/couchdb-stress.git
>
> Don't go running off their yet there's not much to see but some analysis documents and some rough ideas I've thrown together. Please email or add to the project if you can.
>
> I invite you to toss you ideas and suggestion for testing and testing profiles/approaches that you think would be helpful.
>
> I fully expect that there will be several testing files that treat the server in different ways, such as a read only vs logging vs read/write. multiple databases and multiple users on each. Tsung can be run on the same machine as couch, but it can also be placed on a server farm and produce millions of requests for stated periods of time.
>
> It is not my intention to repeat the futon test in tsung, I just used if to scrape of the xml I'll need for writing tests. But the exercise did reveal a few surprises. Some 7 sections failed, likely due to some special handling needed by tsung or maybe some feature in couchdb or timing issues.
>
> Testing was done against today's 1.2 version, please note that the trailing number refers to a page number of the results. I captured the logs of the errors and the calls that were made for those of you who are inclined to investigate. In the next few days I will try to get a sample going of at least one type of test. I'm pretty green with couchdb and tsung so if you have suggestions I'm all ears, I can create specific test suites that reveal warts with your help that hopefully in future will help those developing make couch even better.
>
> Testing recording of futon 1.2.0 with tsung-recorder 1
[snip]
This looks fantastic Mike!!
I'm keen to pitch in and see if we can use this for some
cross-platform benchmarking as well.
A+
Dave