You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by davisp <gi...@git.apache.org> on 2015/07/13 19:18:16 UTC
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
GitHub user davisp opened a pull request:
https://github.com/apache/couchdb-couch/pull/71
Optimize couch_ejson_compare NIF
The old nif was allocating a set of collators that were reserved and
released by each scheduler thread. This coordination and the
accompanying mutex turned into a global point of contention when many
schedulers were using couch_ejson_compare.
This change removes the stack based concurrency control in favor of a
`__thread variable`. We end up with the same number of collators but
without any overhead around locking a central mutex.
This increases performance by roughly a factor of five.
COUCHDB-2732
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloudant/couchdb-couch 2732-optimize-couch-ejson-compare
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/couchdb-couch/pull/71.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #71
----
commit 6b38dfacbb97c5cb7c89a27115d1227c7c52dbba
Author: Paul J. Davis <pa...@gmail.com>
Date: 2015-07-13T17:14:09Z
Optimize couch_ejson_compare NIF
The old nif was allocating a set of collators that were reserved and
released by each scheduler thread. This coordination and the
accompanying mutex turned into a global point of contention when many
schedulers were using couch_ejson_compare.
This change removes the stack based concurrency control in favor of a
`__thread variable`. We end up with the same number of collators but
without any overhead around locking a central mutex.
This increases performance by roughly a factor of five.
COUCHDB-2732
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-couch/pull/71#issuecomment-122002875
Didn't test any performance, but for everything else it works good.
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by davisp <gi...@git.apache.org>.
Github user davisp commented on the pull request:
https://github.com/apache/couchdb-couch/pull/71#issuecomment-121134365
I'm working on getting the actual dataset used and benchmark code. The general outline of the test though is that they loaded something like 35K records into a Q=128 database from the XML music set and then had a view that emitted a row for the artist. The test was then to just run a view query getting 10 artists starting with a random letter between A-O (I assume there were less than 10 after O or something).
This test was then run with a number of different concurrent clients. IIRC somewhere around 100 clients maxed out throughput for both.
The exact numbers we were seeing were about 50-55 req/s with the current NIF implementation and about 340-350 req/s with this TLS version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by robertkowalski <gi...@git.apache.org>.
Github user robertkowalski commented on the pull request:
https://github.com/apache/couchdb-couch/pull/71#issuecomment-121092448
@davisp wowow! how are the numbers for a micro-benchmark that uses of one of our api-endpoints compared to an unpatched version?
I usually run something like this to compare patches:
```
#!/bin/sh
COUCHDB_PATH=$HOME/apache/couchdb
BENCH_RUN="siege -q -c120 -r100 -b http://127.0.0.1:15984/foo/testdoc"
sleep 60;
$COUCHDB_PATH/dev/run --with-admin-party-please & pid=$!
sleep 120
echo ""
echo ""
for i in `seq 1 10`;
do
echo "Runnig test #$i:"
$BENCH_RUN
sleep 90
done
kill $pid
```
and reboot inbetween. i think most important is that i have dropbox and goodle drive and all of their friends plus index services of the OS all disabled
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by rnewson <gi...@git.apache.org>.
Github user rnewson commented on the pull request:
https://github.com/apache/couchdb-couch/pull/71#issuecomment-122001170
bump
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/couchdb-couch/pull/71
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch pull request: Optimize couch_ejson_compare NIF
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-couch/pull/71#issuecomment-121048266
+1
Not sure how to test performance part for now. @robertkowalski do you have anything to fire here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---