You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by gi...@git.apache.org on 2017/05/25 15:25:21 UTC

[GitHub] nickva commented on issue #539: Close idle dbs

nickva commented on issue #539: Close idle dbs
URL: https://github.com/apache/couchdb/pull/539#issuecomment-304039061
 
 
   This is a test restarting a 3 node cluster with 130k databases (100k regular and 30k "_replicator" dbs).
   
   ![close_idle_shards_vs_master_100k_dbs_30k_replicator_dbs](https://cloud.githubusercontent.com/assets/211822/26456179/4f5072d4-413a-11e7-8378-1dfd15918ae7.png)
   
   On the left is master and on the right is this PR. 
   
   There were no requests issued to the cluster during this time. All activity is due to internal replication and replicator. 
   
   Left side crashed and restarted after some point, probably due to high memory usage. 
   
   Notice memory consumption and process count continue to rise after internal sync is finished due to the replicator having to launch change feeds and checkpoint 30k * 8 = 240k shards. Closing idle shards in this case has helped free enough memory to let replicator finish checkpointing all the shards.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services