[GitHub] nickva commented on issue #470: Scheduling Replicator

[gi...@git.apache.org]

nickva commented on issue #470: Scheduling Replicator
URL: https://github.com/apache/couchdb/pull/470#issuecomment-292237042
 
 
   Some additional information from testing how new scheduling replicator handles a large amount of jobs.
   
   This is a graph of adding 1M replication jobs to a cluster of 3 nodes (testy011). Replication jobs were added using dyno/rep tool : https://github.com/cloudant/dyno/blob/master/README_rep.md
   
   ```
   rep.replicate_all_and_compare(n=1000, num=1, normal=False) 
   ```
   
   That creates a connected cluster of replications for for n=1000, it would create 1000*1000=1M replication jobs.
   
   ![db_changes](https://cloud.githubusercontent.com/assets/211822/24765972/c1fb151c-1ac7-11e7-8fd5-45e44d1c4bc2.png)
   
   `couch_replicator.docs.db_changes` tracks the total number of changes seen by the replicator. In this case the replicator has seen the 1M replication documents created above.
   
   ![jobs_total](https://cloud.githubusercontent.com/assets/211822/24766116/231f61e0-1ac8-11e7-9e32-8c95490df4f6.png)
   
   `couch_replicator.jobs.total` tracks the number of replication jobs managed by the scheduler. Out of the 1M jobs, each node in the cluster picked up about 330K each.
   
   ![scheduler](https://cloud.githubusercontent.com/assets/211822/24766148/3daacf4a-1ac8-11e7-9577-60d502321804.png)
   
   `couch_replicator.jobs.starts` counts the number of jobs which have been started by the scheduler. In this case a time derivative was applied. So it shows how the scheduler interval works. Every time it runs it start another 20 jobs (this is specified by the max_churn parameter).
   
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services