You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Ronan Jouchet <ro...@cadensimaging.com> on 2017/01/20 21:21:01 UTC
pouchdb-find / mango: why do queries using `$in` need `_id: {$gt:0}`
?
Hi, this is a mailing-list repost of StackOverflow question
"pouchdb-find: can $in / $or queries use indexes?" [1]
I noticed that queries using $in, such as:
"type": {"$in": ["a", "b"]}
... fail to use the existing relevant index (pouchdb-find warns
"no matching index found, create an index to optimize query time").
StackOverflow user markwatsonatx suggested adding to my query
"_id": {"$gt": 0}
... and indeed I no longer get index warnings with it.
Is this expected behavior? Can anyone explain what's going on? Thanks!
[1]
http://stackoverflow.com/questions/40686383/pouchdb-find-can-in-or-queries-use-indexes
Re: pouchdb-find / mango: why do queries using `$in` need `_id:
{$gt:0}` ?
Posted by Garren Smith <ga...@apache.org>.
Hi Ronan,
I've replied inline.
Cheers
Garren
On Mon, Jan 23, 2017 at 5:34 PM, Ronan Jouchet <
ronan.jouchet@cadensimaging.com> wrote:
> Hi Garren,
>
> I am querying a CouchDB 2.0 backend, using pouchdb-find in
> pure "remote" mode, without any synchronization.
>
> Thanks for the blog post, that helps :) . So, effectively, the
> "_id": {"$gt":0} selector is by no means a way to use an index
> (that's the impression I got when I was suggested to try it).
>
You can use it this way, you just need to understand what it is doing.
If you have a large database this will hurt. But you can use a better value
than 0 to reduce the number of documents that need to be sorted in memory
which is a good thing.
> It's only a way to suppress pouchdb-find's warning, by telling it
> "I know this won't fit a map/reduce, I'll be running in-memory
> operations on allDocs, and I'm aware of the perf. consequences".
>
> Correct?
>
The warning is just to help anyone new to using Mango that what they are
doing isn't the best way on a large database but will be fine on a small
database.
Its a fine way to experiment but once you start noticing performance
issues, creating an index is the way forward.
>
> Final clarification: querying with or without the _id selector
> produces the exact same operations with the same perf and the only
> difference is the warning when we weren't explicit enough, right?
>
Correct
>
> Thanks for the work on pouchdb-find! Good day.
You are welcome, though many other people have contributed more than me.
Its great to hear people using it. Let me know if you have anymore
questions.
>
>
>
> On 2017-01-21 08:12, Garren Smith wrote:
>
>> Hi Ronan,
>>
>> When using pouchdb-find are you querying a CouchDB backend or using
>> PouchDB?
>>
>> The reason that adding "_id": {"$gt": 0} works is because
>> pouchdb-find/mango fetches all the docs using the _all_docs index
>> and then processes the $in operator in memory.
>>
>> We have added functionality that the $in operator should work without
>> needing the "_id": {"$gt": 0} selector. So if it didn't work that
>> means there is a bug in pouchdb-find. I have written a blogpost on
>> how pouchdb-find/mango actually works and it might help you
>> understand what is actually happening [1].
>>
>> Cheers
>>
>> Garren
>>
>> [1]
>> http://www.redcometlabs.com/blog/2015/12/1/a-look-under-the-
>> covers-of-pouchdb-find
>>
>>
>> On Fri, Jan 20, 2017 at 11:21 PM, Ronan Jouchet wrote:
>>
>> Hi, this is a mailing-list repost of StackOverflow question
>>> "pouchdb-find: can $in / $or queries use indexes?" [1]
>>>
>>> I noticed that queries using $in, such as:
>>>
>>> "type": {"$in": ["a", "b"]}
>>>
>>> ... fail to use the existing relevant index (pouchdb-find warns
>>> "no matching index found, create an index to optimize query time").
>>> StackOverflow user markwatsonatx suggested adding to my query
>>>
>>> "_id": {"$gt": 0}
>>>
>>> ... and indeed I no longer get index warnings with it.
>>>
>>> Is this expected behavior? Can anyone explain what's going on?
>>> Thanks!
>>>
>>> [1] http://stackoverflow.com/questions/40686383/pouchdb-find-
>>> can-in-or-queries-use-indexes
>>>
>>>
>>
Re: pouchdb-find / mango: why do queries using `$in` need `_id:
{$gt:0}` ?
Posted by Ronan Jouchet <ro...@cadensimaging.com>.
Hi Garren,
I am querying a CouchDB 2.0 backend, using pouchdb-find in
pure "remote" mode, without any synchronization.
Thanks for the blog post, that helps :) . So, effectively, the
"_id": {"$gt":0} selector is by no means a way to use an index
(that's the impression I got when I was suggested to try it).
It's only a way to suppress pouchdb-find's warning, by telling it
"I know this won't fit a map/reduce, I'll be running in-memory
operations on allDocs, and I'm aware of the perf. consequences".
Correct?
Final clarification: querying with or without the _id selector
produces the exact same operations with the same perf and the only
difference is the warning when we weren't explicit enough, right?
Thanks for the work on pouchdb-find! Good day.
On 2017-01-21 08:12, Garren Smith wrote:
> Hi Ronan,
>
> When using pouchdb-find are you querying a CouchDB backend or using
> PouchDB?
>
> The reason that adding "_id": {"$gt": 0} works is because
> pouchdb-find/mango fetches all the docs using the _all_docs index
> and then processes the $in operator in memory.
>
> We have added functionality that the $in operator should work without
> needing the "_id": {"$gt": 0} selector. So if it didn't work that
> means there is a bug in pouchdb-find. I have written a blogpost on
> how pouchdb-find/mango actually works and it might help you
> understand what is actually happening [1].
>
> Cheers
>
> Garren
>
> [1]
> http://www.redcometlabs.com/blog/2015/12/1/a-look-under-the-covers-of-pouchdb-find
>
>
> On Fri, Jan 20, 2017 at 11:21 PM, Ronan Jouchet wrote:
>
>> Hi, this is a mailing-list repost of StackOverflow question
>> "pouchdb-find: can $in / $or queries use indexes?" [1]
>>
>> I noticed that queries using $in, such as:
>>
>> "type": {"$in": ["a", "b"]}
>>
>> ... fail to use the existing relevant index (pouchdb-find warns
>> "no matching index found, create an index to optimize query time").
>> StackOverflow user markwatsonatx suggested adding to my query
>>
>> "_id": {"$gt": 0}
>>
>> ... and indeed I no longer get index warnings with it.
>>
>> Is this expected behavior? Can anyone explain what's going on?
>> Thanks!
>>
>> [1] http://stackoverflow.com/questions/40686383/pouchdb-find-
>> can-in-or-queries-use-indexes
>>
>
Re: pouchdb-find / mango: why do queries using `$in` need `_id:
{$gt:0}` ?
Posted by Garren Smith <ga...@apache.org>.
Hi Ronan,
When using pouchdb-find are you querying a CouchDB backend or using
PouchDB?
The reason that adding "_id": {"$gt": 0} works is because
pouchdb-find/mango fetches all the docs using the _all_docs index and then
processes the $in operator in memory.
We have added functionality that the $in operator should work without
needing the "_id": {"$gt": 0} selector. So if it didn't work that means
there is a bug in pouchdb-find. I have written a blogpost on how
pouchdb-find/mango actually works and it might help you understand what is
actually happening [1].
Cheers
Garren
[1]
http://www.redcometlabs.com/blog/2015/12/1/a-look-under-the-covers-of-pouchdb-find
On Fri, Jan 20, 2017 at 11:21 PM, Ronan Jouchet <
ronan.jouchet@cadensimaging.com> wrote:
> Hi, this is a mailing-list repost of StackOverflow question
> "pouchdb-find: can $in / $or queries use indexes?" [1]
>
> I noticed that queries using $in, such as:
>
> "type": {"$in": ["a", "b"]}
>
> ... fail to use the existing relevant index (pouchdb-find warns
> "no matching index found, create an index to optimize query time").
> StackOverflow user markwatsonatx suggested adding to my query
>
> "_id": {"$gt": 0}
>
> ... and indeed I no longer get index warnings with it.
>
> Is this expected behavior? Can anyone explain what's going on? Thanks!
>
> [1] http://stackoverflow.com/questions/40686383/pouchdb-find-
> can-in-or-queries-use-indexes
>