[GitHub] garethbowen opened a new issue #1737: Performance regression querying changes using _doc_ids filter

[GitBox <gi...@apache.org>]

garethbowen opened a new issue #1737: Performance regression querying changes using _doc_ids filter
URL: https://github.com/apache/couchdb/issues/1737
 
 
   When upgrading from v1.7.1 to v2.2.0 I noticed our replication was taking longer. I investigated further and found the problem was specifically in relation to the initial request for changes. We use the _doc_ids filter so we can replicate only certain documents and this has been stable and performant on the 1.x versions.
   
   ## Steps to Reproduce
   
   Use [this node script](https://github.com/apache/couchdb/files/2578449/index.js.txt) to create a database and fill it with 1 million docs and then query it for specific IDs.
   
   ## Context
   
   In my testing using the above script I got responses in 1 to 2ms on v1.7.1 and 2500 to 2600ms on 2.2.0.
   In our production database with real world data and about 8 million docs it takes less than a second on v1.7.1 and around 40 seconds on v2.2.0.
   
   This has affected real world performance for users trying to replicate their data.
   
   ## Your Environment
   
   * Version used: 2.2.0 (from 1.7.1)
   * Operating System: Ubuntu 1804
   * Link to your project: https://github.com/medic/medic-webapp/
   
   ## What I've tried
   
   - changing clustering from n=3 q=8 to n=1 q=1 - no improvement
   - changing the `changes_doc_ids_optimization_threshold` at 1, 100, and very large - no improvement
   - using a very large `seq_interval` parameter - no improvement
   - using a mango selector - worse performance

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services