You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by sh...@apache.org on 2017/07/31 07:36:32 UTC
lucene-solr:jira/solr-10821: SOLR-10821: First cut of nav structure
and overview page for autoscaling features in 7.0
Repository: lucene-solr
Updated Branches:
refs/heads/jira/solr-10821 [created] d7f7639fa
SOLR-10821: First cut of nav structure and overview page for autoscaling features in 7.0
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/d7f7639f
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/d7f7639f
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/d7f7639f
Branch: refs/heads/jira/solr-10821
Commit: d7f7639fa1dbe3584f9f1dabb3663a5630b78fcb
Parents: b1a65c8
Author: Shalin Shekhar Mangar <sh...@apache.org>
Authored: Mon Jul 31 13:06:19 2017 +0530
Committer: Shalin Shekhar Mangar <sh...@apache.org>
Committed: Mon Jul 31 13:06:19 2017 +0530
----------------------------------------------------------------------
.../src/solrcloud-autoscaling-api.adoc | 49 ++++++++++++++++
.../src/solrcloud-autoscaling-overview.adoc | 59 ++++++++++++++++++++
...olrcloud-autoscaling-policy-preferences.adoc | 21 +++++++
.../src/solrcloud-autoscaling.adoc | 26 +++++++++
solr/solr-ref-guide/src/solrcloud.adoc | 3 +-
5 files changed, 157 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7f7639f/solr/solr-ref-guide/src/solrcloud-autoscaling-api.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud-autoscaling-api.adoc b/solr/solr-ref-guide/src/solrcloud-autoscaling-api.adoc
new file mode 100644
index 0000000..32cbf82
--- /dev/null
+++ b/solr/solr-ref-guide/src/solrcloud-autoscaling-api.adoc
@@ -0,0 +1,49 @@
+= SolrCloud AutoScaling API
+:page-shortname: solrcloud-autoscaling-api
+:page-permalink: solrcloud-autoscaling-api.html
+:page-toclevels: 2
+:page-tocclass: right
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+The AutoScaling API can be used to read, set and remove cluster policy, preferences, triggers, listeners as well as for diagnostics on the state of the cluster.
+
+== Read API
+
+The AutoScaling read API is available at `/solr/admin/autoscaling` as well as on `/v2/cluster/autoscaling` path. It returns information about the configured cluster preferences, policies, triggers and listeners.
+
+=== Diagnostics
+
+__TODO__
+
+== Write API
+
+=== set-cluster-preferences: Create and modify cluster preferences
+
+__TODO__
+
+=== set-cluster-policy: Create and modify cluster preferences
+
+__TODO__
+
+=== set-policy: Create and modify collection specific policy
+
+__TODO__
+
+=== remove-policy: Remove a collection specific policy
+
+__TODO__
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7f7639f/solr/solr-ref-guide/src/solrcloud-autoscaling-overview.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud-autoscaling-overview.adoc b/solr/solr-ref-guide/src/solrcloud-autoscaling-overview.adoc
new file mode 100644
index 0000000..dac4745
--- /dev/null
+++ b/solr/solr-ref-guide/src/solrcloud-autoscaling-overview.adoc
@@ -0,0 +1,59 @@
+= Overview of AutoScaling in SolrCloud
+:page-shortname: solrcloud-autoscaling-overview
+:page-permalink: solrcloud-autoscaling-overview.html
+:page-toclevels: 1
+:page-tocclass: right
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+The goal of autoscaling feature is to make SolrCloud cluster management easier, automatic and intelligent. It aims to provide good defaults such that the cluster remains balanced and stable in the face of various events such as a node joining the cluster or leaving the cluster. This is achieved by satisfying a set of rules and sorting preferences that help Solr select the target of cluster management operations.
+
+== Cluster Preferences
+
+The cluster preferences, as the name suggests, apply to all cluster management operations regardless of which collection they affect.
+
+A preference is a sort conditions that help Solr select nodes that either maximize or minimize given metrics e.g. a preference `{minimize : cores}` will help Solr select nodes such that the number of cores on each node is minimized. We write cluster preference in a way that reduces the overall load on the system. You can add more than one preferences to break ties.
+
+The default cluster preferences consist of the above example which is to minimize the number of cores on all nodes.
+
+You can learn more about preferences in the __TODO__ section.
+
+== Cluster Policy
+
+The cluster policy is a set of conditions that a node, shard or collection must satisfy before it can be chosen as the target of a cluster management operation. These conditions are applied across the cluster regardless of the collection being managed. For example, the condition `{"cores":"<10", "node":"#ANY"}` means that any node must have less than ten Solr cores in total regardless of which collection they belong to.
+
+There are many metrics on which the condition can be based e.g. system load average, heap usage, free disk space etc. The full list of supported metrics can be found at __TODO__ section.
+
+When a node, shard or collection does not satisfy the policy, we call it a *violation*. Solr ensures that cluster management operations minimize the number of violations. The cluster management operations are either invoked manually by us or automatically in response to *Triggers*.
+
+== Collection specific policies
+
+Sometimes a collection may need conditions in addition to those specified in the cluster policy. In such cases, we can create named policies that can be used for specific collections. Firstly, we can use the `set-policy` API to create a new policy and then specify `policy=<policy_name>` parameter to the create collection API.
+
+`http://localhost:8983/solr/admin/collections?action=CREATE&name=coll1&numShards=1&replicationFactor=2&policy=policy1`
+
+The above create collection command will associate a policy named policy1 with the collection named coll1. Only a single policy may be associated with a collection.
+
+Note that the collection specific policy is applied *in addition* to the cluster policy i.e. it is not an override but an augmentation. Therefore the collection will follow all conditions laid out in the cluster preferences, cluster policy and the policy named `policy1`.
+
+You can learn more about collection specific policies in the __TODO__ section.
+
+== AutoScaling APIs
+
+The autoscaling APIs available at `/admin/autoscaling` can be used to read and modify each of the components discussed above.
+
+You can learn more about these APIs in the __TODO__ section.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7f7639f/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc b/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
new file mode 100644
index 0000000..bc87fc3
--- /dev/null
+++ b/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
@@ -0,0 +1,21 @@
+= SolrCloud AutoScaling Policy and Preferences
+:page-shortname: solrcloud-autoscaling-policy-preferences
+:page-permalink: solrcloud-autoscaling-policy-preferences.html
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+TODO
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7f7639f/solr/solr-ref-guide/src/solrcloud-autoscaling.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud-autoscaling.adoc b/solr/solr-ref-guide/src/solrcloud-autoscaling.adoc
new file mode 100644
index 0000000..373818d
--- /dev/null
+++ b/solr/solr-ref-guide/src/solrcloud-autoscaling.adoc
@@ -0,0 +1,26 @@
+= SolrCloud AutoScaling
+:page-shortname: solrcloud-autoscaling
+:page-permalink: solrcloud-autoscaling.html
+:page-children: solrcloud-autoscaling-overview, solrcloud-autoscaling-api, solrcloud-autoscaling-policy-preferences
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+This section describes the autoscaling features of SolrCloud. It covers the following topics:
+
+* <<solrcloud-autoscaling-overview.adoc#solrcloud-autoscaling-overview,Overview of AutoScaling in SolrCloud>>
+* <<solrcloud-autoscaling-api.adoc#solrcloud-autoscaling-api,SolrCloud AutoScaling API>>
+* <<solrcloud-autoscaling-policy-preferences.adoc#solrcloud-autoscaling-policy-preferences,SolrCloud AutoScaling Policy and Preferences>>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7f7639f/solr/solr-ref-guide/src/solrcloud.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud.adoc b/solr/solr-ref-guide/src/solrcloud.adoc
index db59ada..36154a2 100644
--- a/solr/solr-ref-guide/src/solrcloud.adoc
+++ b/solr/solr-ref-guide/src/solrcloud.adoc
@@ -1,7 +1,7 @@
= SolrCloud
:page-shortname: solrcloud
:page-permalink: solrcloud.html
-:page-children: getting-started-with-solrcloud, how-solrcloud-works, solrcloud-configuration-and-parameters, rule-based-replica-placement, cross-data-center-replication-cdcr
+:page-children: getting-started-with-solrcloud, how-solrcloud-works, solrcloud-configuration-and-parameters, rule-based-replica-placement, cross-data-center-replication-cdcr, solrcloud-autoscaling
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
@@ -45,3 +45,4 @@ In this section, we'll cover everything you need to know about using Solr in Sol
** <<configsets-api.adoc#configsets-api,ConfigSets API>>
* <<rule-based-replica-placement.adoc#rule-based-replica-placement,Rule-based Replica Placement>>
* <<cross-data-center-replication-cdcr.adoc#cross-data-center-replication-cdcr,Cross Data Center Replication (CDCR)>>
+*<<solrcloud-autoscaling.adoc#solrcloud-autoscaling,SolrCloud AutoScaling>>