You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Nick Poulden <ni...@domine.co.uk> on 2010/03/15 20:23:34 UTC
Performance when sending array of keys to a view
Hi,
I'm developing a web app with node.js and CouchDB that displays nearby
restaurants when given a lat/lng. At the moment I'm storing the restaurant
data in CouchDB and a MySQL table with couchdb-key / lat / lng for each
restaurant. I query MySQL for the nearby restaurants, then do a POST to a
CouchDB view with the keys returned by MySQL.
The problem is that there seems to be a huge difference in performance when
sending the list of keys compared to a 'normal' view query.
Here's an example:
var db = CouchDB.db('restaurants');
profiles.push(["Before Couch request: ", Number(new Date)])
db.view('Restaurant/by_idx', {keys: restaurant_keys, include_docs: true,
success: function(response) {
profiles.push(["After couch result: ", Number(new Date)])
}});
Start: 0 ms
Before MySQL connect: 0 ms
Got MySQL result: 17 ms
Before Couch request: 18 ms
After couch result: 272 ms
If I replace line 3 with this:
db.view('Restaurant/by_idx', {limit: 20, include_docs: true, success:
function(response) {
I get this:
Start: 0ms
Before MySQL connect: 0 ms
Got MySQL result: 12 ms
Before Couch request: 13 ms
After couch result: 21 ms
In both cases there are about 20 restaurants returned. The database has
about 500 documents in it altogether. Can anyone shed some light on why
there is such a big difference in response time?
Thanks,
Nick
Re: Performance when sending array of keys to a view
Posted by Jan Lehnardt <ja...@apache.org>.
Can you point out the error so we can get it fixed in the library?
Cheers
Jan
--
On 15 Mar 2010, at 12:50, Nick Poulden wrote:
> After further investigation the problem was with the CouchDB node plugin.
> Using pure node.js it's nice and fast. Thanks for the help :-)
>
> Nick
>
> On Mon, Mar 15, 2010 at 12:33 PM, Paul Davis <pa...@gmail.com>wrote:
>
>> When you post a keys array its doing a search through the view btree
>> to get each key. Where as with limit=20, it's searching once and then
>> scanning linearly.
>>
>> What happens if you sort the array of keys before posting them?
>>
>> On Mon, Mar 15, 2010 at 3:23 PM, Nick Poulden <ni...@domine.co.uk> wrote:
>>> Hi,
>>>
>>> I'm developing a web app with node.js and CouchDB that displays nearby
>>> restaurants when given a lat/lng. At the moment I'm storing the
>> restaurant
>>> data in CouchDB and a MySQL table with couchdb-key / lat / lng for each
>>> restaurant. I query MySQL for the nearby restaurants, then do a POST to a
>>> CouchDB view with the keys returned by MySQL.
>>>
>>> The problem is that there seems to be a huge difference in performance
>> when
>>> sending the list of keys compared to a 'normal' view query.
>>>
>>> Here's an example:
>>>
>>> var db = CouchDB.db('restaurants');
>>> profiles.push(["Before Couch request: ", Number(new Date)])
>>> db.view('Restaurant/by_idx', {keys: restaurant_keys, include_docs: true,
>>> success: function(response) {
>>> profiles.push(["After couch result: ", Number(new Date)])
>>> }});
>>>
>>> Start: 0 ms
>>> Before MySQL connect: 0 ms
>>> Got MySQL result: 17 ms
>>> Before Couch request: 18 ms
>>> After couch result: 272 ms
>>>
>>> If I replace line 3 with this:
>>>
>>> db.view('Restaurant/by_idx', {limit: 20, include_docs: true, success:
>>> function(response) {
>>>
>>> I get this:
>>>
>>> Start: 0ms
>>> Before MySQL connect: 0 ms
>>> Got MySQL result: 12 ms
>>> Before Couch request: 13 ms
>>> After couch result: 21 ms
>>>
>>> In both cases there are about 20 restaurants returned. The database has
>>> about 500 documents in it altogether. Can anyone shed some light on why
>>> there is such a big difference in response time?
>>>
>>> Thanks,
>>>
>>> Nick
>>>
>>
Re: Performance when sending array of keys to a view
Posted by Nick Poulden <ni...@domine.co.uk>.
After further investigation the problem was with the CouchDB node plugin.
Using pure node.js it's nice and fast. Thanks for the help :-)
Nick
On Mon, Mar 15, 2010 at 12:33 PM, Paul Davis <pa...@gmail.com>wrote:
> When you post a keys array its doing a search through the view btree
> to get each key. Where as with limit=20, it's searching once and then
> scanning linearly.
>
> What happens if you sort the array of keys before posting them?
>
> On Mon, Mar 15, 2010 at 3:23 PM, Nick Poulden <ni...@domine.co.uk> wrote:
> > Hi,
> >
> > I'm developing a web app with node.js and CouchDB that displays nearby
> > restaurants when given a lat/lng. At the moment I'm storing the
> restaurant
> > data in CouchDB and a MySQL table with couchdb-key / lat / lng for each
> > restaurant. I query MySQL for the nearby restaurants, then do a POST to a
> > CouchDB view with the keys returned by MySQL.
> >
> > The problem is that there seems to be a huge difference in performance
> when
> > sending the list of keys compared to a 'normal' view query.
> >
> > Here's an example:
> >
> > var db = CouchDB.db('restaurants');
> > profiles.push(["Before Couch request: ", Number(new Date)])
> > db.view('Restaurant/by_idx', {keys: restaurant_keys, include_docs: true,
> > success: function(response) {
> > profiles.push(["After couch result: ", Number(new Date)])
> > }});
> >
> > Start: 0 ms
> > Before MySQL connect: 0 ms
> > Got MySQL result: 17 ms
> > Before Couch request: 18 ms
> > After couch result: 272 ms
> >
> > If I replace line 3 with this:
> >
> > db.view('Restaurant/by_idx', {limit: 20, include_docs: true, success:
> > function(response) {
> >
> > I get this:
> >
> > Start: 0ms
> > Before MySQL connect: 0 ms
> > Got MySQL result: 12 ms
> > Before Couch request: 13 ms
> > After couch result: 21 ms
> >
> > In both cases there are about 20 restaurants returned. The database has
> > about 500 documents in it altogether. Can anyone shed some light on why
> > there is such a big difference in response time?
> >
> > Thanks,
> >
> > Nick
> >
>
Re: Performance when sending array of keys to a view
Posted by Paul Davis <pa...@gmail.com>.
When you post a keys array its doing a search through the view btree
to get each key. Where as with limit=20, it's searching once and then
scanning linearly.
What happens if you sort the array of keys before posting them?
On Mon, Mar 15, 2010 at 3:23 PM, Nick Poulden <ni...@domine.co.uk> wrote:
> Hi,
>
> I'm developing a web app with node.js and CouchDB that displays nearby
> restaurants when given a lat/lng. At the moment I'm storing the restaurant
> data in CouchDB and a MySQL table with couchdb-key / lat / lng for each
> restaurant. I query MySQL for the nearby restaurants, then do a POST to a
> CouchDB view with the keys returned by MySQL.
>
> The problem is that there seems to be a huge difference in performance when
> sending the list of keys compared to a 'normal' view query.
>
> Here's an example:
>
> var db = CouchDB.db('restaurants');
> profiles.push(["Before Couch request: ", Number(new Date)])
> db.view('Restaurant/by_idx', {keys: restaurant_keys, include_docs: true,
> success: function(response) {
> profiles.push(["After couch result: ", Number(new Date)])
> }});
>
> Start: 0 ms
> Before MySQL connect: 0 ms
> Got MySQL result: 17 ms
> Before Couch request: 18 ms
> After couch result: 272 ms
>
> If I replace line 3 with this:
>
> db.view('Restaurant/by_idx', {limit: 20, include_docs: true, success:
> function(response) {
>
> I get this:
>
> Start: 0ms
> Before MySQL connect: 0 ms
> Got MySQL result: 12 ms
> Before Couch request: 13 ms
> After couch result: 21 ms
>
> In both cases there are about 20 restaurants returned. The database has
> about 500 documents in it altogether. Can anyone shed some light on why
> there is such a big difference in response time?
>
> Thanks,
>
> Nick
>