[jira] [Commented] (SOLR-13831) Support defining arbitrary autoscaling simulation scenarios

["Andrzej Bialecki (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951113#comment-16951113 ] 

Andrzej Bialecki commented on SOLR-13831:
-----------------------------------------

Patch with an implementation of {{SimScenario}} tool for running scenarios. This tool is hooked up to the {{bin/solr autoscaling -scenario <path>}} .

Functionality supported in this patch:
 * cluster and collection setup tasks, including initialization from autoscaling snapshots or from live clusters
 * ability to process some SolrRequests (the same subset as supported in {{SimCloudManager}})
 * primitive loops
 * adding / killing nodes
 * calculating and applying autoscaling suggestions
 * adjusting delays between operations to simulate slow actions
 * variable substitution in task params
 * listening to trigger events
 * saving intermediate and final autoscaling snapshots, with optional redaction
 * dumping cluster state and autoscaling data to console, with optional redaction

 

Complete list of tasks supported in this patch:
{code:java}
 public enum SimAction {
  CREATE_CLUSTER,
  LOAD_SNAPSHOT,
  SAVE_SNAPSHOT,
  CALCULATE_SUGGESTIONS,
  APPLY_SUGGESTIONS,
  KILL_NODES,
  ADD_NODES,
  LOAD_AUTOSCALING,
  LOOP_START,
  LOOP_END,
  SET_OP_DELAYS,
  SOLR_REQUEST,
  WAIT_COLLECTION,
  SET_LISTENER,
  WAIT_LISTENER,
  RUN,
  DUMP,
  CTX_SET,
  CTX_REMOVE;
{code}
Example scenario DSL:
{code:java}
# standard comment
// java comment
create_cluster numNodes=2 // inline comment
load_autoscaling json={'cluster-policy'+:+[{'replica'+:+'<3',+'shard'+:+'#EACH',+'collection'+:+'testCollection','node':'#ANY'}]}&defaultWaitFor=10
solr_request /admin/collections?action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
wait_collection collection=testCollection&shards=2&replicas=2
set_listener trigger=.auto_add_replicas&stage=SUCCEEDED
kill_nodes numNodes=1
wait_listener trigger=.auto_add_replicas&wait=60
wait_collection collection=testCollection&shards=2&replicas=2
save_snapshot path=${snapshotPath} {code}
 

Another example with a loop:
{code:java}
load_snapshot path=${existingSnapshotPath}
solr_request /admin/collections?action=CREATE&autoAddReplicas=true&name=testCollection&numShards=2&replicationFactor=2&maxShardsPerNode=2
wait_collection collection=testCollection&shards=2&replicas=2
ctx_set key=myNode&value=${_random_node_}
solr_request /admin/collections?action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
solr_request /admin/collections?action=ADDREPLICA&collection=testCollection&shard=shard1&node=${myNode}
loop_start iterations=10
  calculate_suggestions
  apply_suggestions
  save_snapshot path=${snapshotPath}/${loopIter}
loop_end
dump redact=true{code}
 

> Support defining arbitrary autoscaling simulation scenarios
> -----------------------------------------------------------
>
>                 Key: SOLR-13831
>                 URL: https://issues.apache.org/jira/browse/SOLR-13831
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Andrzej Bialecki
>            Assignee: Andrzej Bialecki
>            Priority: Major
>         Attachments: SOLR-13831.patch
>
>
> In many cases where the {{bin/solr autoscaling}} tool is used it would be very useful to be able to specify a concrete scenario to play out, eg:
>  * load a snapshot (or create a simulated cluster of N nodes)
>  * calculate suggestions
>  * apply suggestions
>  * kill one or more nodes
>  * loop N times
>  * make some arbitrary SolrRequest-s
>  * save snapshot
>  * etc...
>  
> This could be expressed as a very simple DSL that can be loaded from a text file, with the following format:
> {code:java}
> # comments
> // or comments
> create_cluster numNodes=5 // inline comment
> solr_request /admin/collections?action=CREATE&name=testCollection&numShards=2&replicationFactor=2
> loop_start iterations=10
>   calculate_suggestions
>   apply_suggestions
> loop_end
> save_snapshot path=/foo{code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org