You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by sh...@apache.org on 2018/03/16 18:09:15 UTC
lucene-solr:branch_7x: SOLR-12097: Document the diskType policy
attribute and usage of disk space in Collection APIs
Repository: lucene-solr
Updated Branches:
refs/heads/branch_7x fd27545a1 -> d7d278c44
SOLR-12097: Document the diskType policy attribute and usage of disk space in Collection APIs
(cherry picked from commit 4c8825b)
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/d7d278c4
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/d7d278c4
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/d7d278c4
Branch: refs/heads/branch_7x
Commit: d7d278c44abc1f63983cc5678035daf00391bb96
Parents: fd27545
Author: Shalin Shekhar Mangar <sh...@apache.org>
Authored: Fri Mar 16 23:38:35 2018 +0530
Committer: Shalin Shekhar Mangar <sh...@apache.org>
Committed: Fri Mar 16 23:39:07 2018 +0530
----------------------------------------------------------------------
solr/CHANGES.txt | 3 +++
solr/solr-ref-guide/src/collections-api.adoc | 10 ++++++++++
.../src/solrcloud-autoscaling-policy-preferences.adoc | 14 +++++++++++---
3 files changed, 24 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7d278c4/solr/CHANGES.txt
----------------------------------------------------------------------
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index 082719d..b2514ef 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -432,6 +432,9 @@ Other Changes
* SOLR-12098: Document the Lucene spins auto-detection and its effect on CMS dynamic defaults.
(Cassandra Targett, shalin)
+* SOLR-12097: Document the diskType policy attribute and usage of disk space in Collection APIs.
+ (Cassandra Targett, shalin)
+
================== 7.2.1 ==================
Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7d278c4/solr/solr-ref-guide/src/collections-api.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/collections-api.adoc b/solr/solr-ref-guide/src/collections-api.adoc
index 7fdee52..2f1ce2e 100644
--- a/solr/solr-ref-guide/src/collections-api.adoc
+++ b/solr/solr-ref-guide/src/collections-api.adoc
@@ -245,6 +245,10 @@ This command allows for seamless splitting and requires no downtime. A shard bei
The split is performed by dividing the original shard's hash range into two equal partitions and dividing up the documents in the original shard according to the new sub-ranges. Two parameters discussed below, `ranges` and `split.key` provide further control over how the split occurs.
+The newly created shards will have as many replicas as the parent shard.
+
+You must ensure that the node running the leader of the parent shard has enough free disk space i.e. more than twice the index size, for the split to succeed. The API uses the Autoscaling framework to find nodes that can satisfy the disk requirements for the new replicas but only when an Autoscaling policy is configured. Refer to <<solrcloud-autoscaling-policy-preferences.adoc#solrcloud-autoscaling-policy-preferences,Autoscaling Policy and Preferences>> section for more details.
+
Shard splitting can be a long running process. In order to avoid timeouts, you should run this as an <<Asynchronous Calls,asynchronous call>>.
=== SPLITSHARD Parameters
@@ -960,6 +964,8 @@ http://localhost:8983/solr/admin/collections?action=DELETEREPLICA&collection=tes
Add a replica to a shard in a collection. The node name can be specified if the replica is to be created in a specific node.
+The API uses the Autoscaling framework to find nodes that can satisfy the disk requirements for the new replica but only when an Autoscaling policy is configured. Refer to <<solrcloud-autoscaling-policy-preferences.adoc#solrcloud-autoscaling-policy-preferences,Autoscaling Policy and Preferences>> section for more details.
+
`/admin/collections?action=ADDREPLICA&collection=_collection_&shard=_shard_&node=_nodeName_`
=== ADDREPLICA Parameters
@@ -2159,6 +2165,8 @@ This command recreates replicas in one node (the source) to another node(s) (the
For source replicas that are also shard leaders the operation will wait for the number of seconds set with the `timeout` parameter to make sure there's an active replica that can become a leader (either an existing replica becoming a leader or the new replica completing recovery and becoming a leader).
+The API uses the Autoscaling framework to find nodes that can satisfy the disk requirements for the new replicas but only when an Autoscaling policy is configured. Refer to <<solrcloud-autoscaling-policy-preferences.adoc#solrcloud-autoscaling-policy-preferences,Autoscaling Policy and Preferences>> section for more details.
+
`/admin/collections?action=REPLACENODE&sourceNode=_source-node_&targetNode=_target-node_`
=== REPLACENODE Parameters
@@ -2188,6 +2196,8 @@ This operation does not hold necessary locks on the replicas that belong to on t
This command moves a replica from one node to a new node. In case of shared filesystems the `dataDir` will be reused.
+The API uses the Autoscaling framework to find nodes that can satisfy the disk requirements for the replica to be moved but only when an Autoscaling policy is configured. Refer to <<solrcloud-autoscaling-policy-preferences.adoc#solrcloud-autoscaling-policy-preferences,Autoscaling Policy and Preferences>> section for more details.
+
`/admin/collections?action=MOVEREPLICA&collection=collection&shard=shard&replica=replica&sourceNode=nodeName&targetNode=nodeName`
=== MOVEREPLICA Parameters
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d7d278c4/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc b/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
index c38b2d8..6fd0e78 100644
--- a/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
+++ b/solr/solr-ref-guide/src/solrcloud-autoscaling-policy-preferences.adoc
@@ -139,6 +139,11 @@ Any arbitrary system property set on the node on startup.
`metrics:<full-path-to-the metric>`::
Any arbitrary metric. For example, `metrics:solr.node:CONTAINER.fs.totalSpace`. Refer to the `key` parameter in the <<metrics-reporting.adoc#metrics-reporting, Metrics API>> section.
+`diskType`::
+The type of disk drive being used for Solr's `coreRootDirectory`. The only two supported values are `rotational` and `ssd`. Refer to `coreRootDirectory` parameter in the <<format-of-solr-xml.adoc#solr-xml-parameters, Solr.xml Parameters>> section.
++
+It's value is fetched from the Metrics API with the key named `solr.node:CONTAINER.fs.coreRoot.spins`. The disk type is auto-detected by Lucene using various heuristics and it is not guaranteed to be correct across all platforms or operating systems. Refer to the <<taking-solr-to-production.adoc#dynamic-defaults-for-concurrentmergescheduler, Dynamic defaults for ConcurrentMergeScheduler>> section for more details.
+
=== Policy Operators
Each attribute in the policy may specify one of the following operators along with the value.
@@ -192,12 +197,15 @@ Place all replicas in nodes with freedisk more than 500GB when possible. Here we
[source,json]
{"replica": 0, "freedisk": "<500", "strict" : false}
-==== Try to Place all Replicas of type TLOG in SSD type file system
+==== Try to Place all Replicas of type TLOG in nodes with SSD drives
[source,json]
-{ "replica": 0, "sysprop.file_system" : "!ssd", "type" : "TLOG" }
+{ "replica": 0, "diskType" : "!ssd", "type" : "TLOG" }
-Please note that to use the `sysprop.fs` attribute all your ssd nodes must be started with a system property `-Dfile_system=ssd`.
+==== Try to Place all Replicas of type PULL on nodes with rotational disk drives
+
+[source,json]
+{ "replica": 0, "diskType" : "!rotational", "type" : "PULL" }
[[collection-specific-policy]]
== Defining Collection-Specific Policies