[GitHub] nickva commented on issue #1621: Black matter in shard files

[GitBox <gi...@apache.org>]

nickva commented on issue #1621: Black matter in shard files
URL: https://github.com/apache/couchdb/issues/1621#issuecomment-424007800
 
 
   @adrienverge 
   
   Thanks for your report. A lot of local documents could increase the file size like that.
   
   Do you know which type you have more of _local/<id> or _local/shard-sync-<id>?
   
   If it is the first kind (_local/<id>), then those are the result of using the database as a replication endpoint.  CouchDB writes _local checkpoint documents on the source and target, so next time it restarts the replication between source and target it will resume from changes feed from the checkpointed value and not from the very beginning. 
   
   However that means that each {source, target} pair (along with a few other thing like if there a filter) generates a _local document. If say source is used for a public demo, then there could be a very large number of {source, target} pairs generated and thus a large number of _local/<id> documents present. 
   
   There is an option to avoid this http://docs.couchdb.org/en/stable/config/replicator.html#replicator/use_checkpoints and it could help in these cases. Read additional info and the warning regarding using that option, but it sounds like it might help your case.
   
   If you see more of the second kind (_local/shard-sync-<id>), then it is hard to say without more debugging why that's happening.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services