You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "zmara (via GitHub)" <gi...@apache.org> on 2024/08/21 12:08:12 UTC

[I] $in and $or too slow for large database (doesn´t use index) [couchdb]

zmara opened a new issue, #5189:
URL: https://github.com/apache/couchdb/issues/5189

   My database has 4.7GB of data (3191269 docs)
   Attribute _id has implicit index, this selector works well:
   
   `{
      "selector": {
         "_id": {
            "$eq": "CustomerElastic_00001280-7a18-466b-9e9c-da5a2d903bda"
         }
      }
   }`
   
   This selector is pending and never finished (the same when I use $or):
   
   `{
      "selector": {
         "_id": {
            "$in": ["CustomerElastic_00000a29-329f-477f-8d4a-8244a9c738aa", "CustomerElastic_00001280-7a18-466b-9e9c-da5a2d903bda"]
         }
      }
   }`
   
   Why?
   
   [NOTE]: # ( Describe the problem you're encountering. )
   [TIP]:  # ( Do NOT give us access or passwords to your actual CouchDB! )
   
   ## Steps to Reproduce
   
   Use fauxton and mango query.
   
   ## Expected Behaviour
   
   It should work the same as $eq
   
   ## Your Environment
   
   {"couchdb":"Welcome","version":"3.3.2","git_sha":"11a234070","uuid":"ff2fae1b3089d86fd741e84a6e532eda","features":["access-ready","partitioned","pluggable-storage-engines","reshard","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
   
   * Browser name and version: Chrome latest version
   * Operating system and version: Windows 11
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] $in and $or too slow for large database (doesn´t use index) [couchdb]

Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #5189:
URL: https://github.com/apache/couchdb/issues/5189#issuecomment-2307302404

   I recommend building a map-reduce view directly (queried via `_view` not `_find`). This is an actual index (on the emitted key), and any query you can do to `_view` will be done efficiently (`startkey`/`endkey` or `key` or `keys`) .
   
   All that `_find` does is look at all the indexes and tries to find the most efficient one for the selector. If it doesn't find one it just reads the whole database and filters out non-matches.
   
   In your case you could do the much more efficient `/dbname/_all_docs?keys=["CustomerElastic_00000a29-329f-477f-8d4a-8244a9c738aa", "CustomerElastic_00001280-7a18-466b-9e9c-da5a2d903bda"]` instead.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] $in and $or too slow for large database (doesn´t use index) [couchdb]

Posted by "zmara (via GitHub)" <gi...@apache.org>.
zmara commented on issue #5189:
URL: https://github.com/apache/couchdb/issues/5189#issuecomment-2303909674

   Thanks for explanation. What about that $in operator? I am using it on one field, so what is the issue there?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] $in and $or too slow for large database (doesn´t use index) [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5189:
URL: https://github.com/apache/couchdb/issues/5189#issuecomment-2303538445

   Thanks for your report @zmara. Yeah it's a known deficiency that`$or` doesn't use an index. It would have to effectively use use the two or indexes and merge them appropriately, and that part isn't implemented currently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org