[GitHub] evsukov89 commented on issue #831: Filter _changes by key to speed up replication

[GitBox <gi...@apache.org>]

evsukov89 commented on issue #831: Filter _changes by key to speed up replication
URL: https://github.com/apache/couchdb/issues/831#issuecomment-405090786
 
 
   @whoali lets take an example: you have an app that uses couchdb as the main data storage. It is intended to be used with offline-first clients (PouchDB/CouchBase Lite 1.x). Let’s assume that overall there is about 1k docs to replicate.
   
   Generally speaking you have two options:
   1. on the server create a new “proxy” database, replicate this 1k docs there and finally use it as a source from offline clients to replicate from
   2. use filtered replication
   
   Analyzing the consequences of each solution:
   1. added complexity, data duplication, additional server usage resources - in my understanding per-user db was at least partiality made to addresss this problem, but I think the shortcomings are known by now 
   2. since the _changes feed is not indexed, what is the CPU+IO cost of filtering 1k docs over the db of 1M docs? what if you have 1k users, 10k, 100k?
   
   @redgeoff we have a patch of indexed changes implementation based on views which works well for us, including filtering by key, etc. We can share the details if you wanna wish.
   
   I do however agree this approach has a few shortcomings: view and _changes outputs are not exactly idempotent. For ex: if the view no longer emits the doc, how this should be reflected in the _changes feed?
   
   @janl @whoali I’m sorry for my ignorance, but would you mind pointing me to the page explaining the process of submitting proposals for couchdb?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services