[jira] [Commented] (SOLR-13933) Cluster mode Stress test suite

["Ishan Chattopadhyaya (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173556#comment-17173556 ] 

Ishan Chattopadhyaya commented on SOLR-13933:
---------------------------------------------

I am actively working on this here: https://github.com/TheSearchStack/solr-bench/tree/stress-harness

Here is a sample suite: https://github.com/TheSearchStack/solr-bench/blob/stress-harness/rolling.json
It starts 3 Solr nodes, 1GB ram each.
It has two tasks: 
# task1 that creates a bunch of collections and indexes some data in them.
# task2 that waits for task2 to end, and after that it does a rolling restart of the cluster (i.e. restarts a node and waits until all replicas in the restarted node is active, and proceeds to the next node until all nodes are restarted).

While doing this, it measures the timings on all of them.

Here is another sample suite: https://github.com/TheSearchStack/solr-bench/blob/stress-harness/workflow.json

Does various tasks in task1 through task5, including indexing, creating many collections, shard splitting, restarting a node, validating number of documents in a collection (this one is WIP). As defined, some of these tasks are done in parallel with each other, some are blocking tasks.


It is my intention to quickly wrap up this quite and start running automated tests for medium scale Solr cluster (say, 50-100 nodes, ~2GB RAM each) on each of our branches and publishing results periodically. This will be specially useful to benchmark stability of the cluster, with SOLR-13951 and SOLR-14636.

> Cluster mode Stress test suite 
> -------------------------------
>
>                 Key: SOLR-13933
>                 URL: https://issues.apache.org/jira/browse/SOLR-13933
>             Project: Solr
>          Issue Type: Bug
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Ishan Chattopadhyaya
>            Priority: Major
>
> We need a stress test harness based on 10s or 100s of nodes, 1000s of collection API operations, overseer operations etc. This suite should run nightly, publish results publicly, so as to help with:
> # Uncover stability problems
> # Benchmarking (timings, resource metrics etc.) on collection operations
> # Indexing/querying performance
> # Validate the accuracy of potential improvements
> References:
> SOLR-10317
> https://github.com/lucidworks/solr-scale-tk
> https://github.com/shalinmangar/solr-perf-tools
> Lucene benchmarks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org