[15/58] [abbrv] lucene-solr:jira/solr-10233: squash merge jira/solr-10290 into master

[tf...@apache.org]

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/metrics-reporting.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/metrics-reporting.adoc b/solr/solr-ref-guide/src/metrics-reporting.adoc
new file mode 100644
index 0000000..24a4ee1
--- /dev/null
+++ b/solr/solr-ref-guide/src/metrics-reporting.adoc
@@ -0,0 +1,264 @@
+= Metrics Reporting
+:page-shortname: metrics-reporting
+:page-permalink: metrics-reporting.html
+
+Solr includes a developer API and instrumentation for the collection of detailed performance-oriented metrics throughout the life-cycle of Solr service and its various components.
+
+Internally this feature uses the http://metrics.dropwizard.io[Dropwizard Metrics API], which uses the following classes of meters to measure events:
+
+* *counters* - simply count events. They provide a single long value, e.g., the number of requests.
+* *meters* - additionally compute rates of events. Provide a count (as above) and 1-, 5-, and 15-minute exponentially decaying rates, similar to the Unix system load average.
+* *histograms* - calculate approximate distribution of events according to their values. Provide the following approximate statistics, with a similar exponential decay as above: mean (arithmetic average), median, maximum, minimum, standard deviation, and 75^th^, 95^th^, 98^th^, 99^th^ and 999^th^ percentiles.
+* *timers* - measure the number and duration of events. They provide a count and histogram of timings.
+* *gauges* - offer instantaneous reading of a current value, e.g., current queue depth, current number of active connections, free heap size.
+
+Each group of related metrics with unique names is managed in a *metric registry*. Solr maintains several such registries, each corresponding to a high-level group such as: `jvm`, `jetty`, `http`, `node`, and `core` (see <<Metric Registries>> below).
+
+For each group (and/or for each registry) there can be several *reporters*, which are components responsible for communication of metrics from selected registries to external systems. Currently implemented reporters support emitting metrics via JMX, Ganglia, Graphite and SLF4J.
+
+There is also a dedicated `/admin/metrics` handler that can be queried to report all or a subset of the current metrics from multiple registries.
+
+== Metric Registries
+
+Solr includes multiple metric registries, which group related metrics.
+
+Metrics are maintained and accumulated through all lifecycles of components from the start of the process until its shutdown - e.g., metrics for a particular SolrCore are tracked through possibly several load, unload and/or rename operations, and are deleted only when a core is explicitly deleted. However, metrics are not persisted across process restarts; restarting Solr will discard all collected metrics.
+
+These are the major groups of metrics that are collected:
+
+=== JVM Registry (`solr.jvm`):
+
+* direct and mapped buffer pools
+* class loading / unloading
+* OS memory, CPU time, file descriptors, swap, system load
+* GC count and time
+* heap, non-heap memory and GC pools
+* number of threads, their states and deadlocks
+
+=== Node / CoreContainer Registry (`solr.node`):
+
+* handler requests (count, timing): collections, info, admin, configSets, etc.
+* number of cores (loaded, lazy, unloaded)
+
+=== Core (SolrCore) Registry:
+
+The <<Core Level Metrics,Core (SolrCore) Registry>> includes `solr.core.<collection>`, one for each core.
+
+* all common RequestHandler-s report: request timers / counters, timeouts, errors.
+* <<Index Merge Metrics,index-level events>>: meters for minor / major merges, number of merged docs, number of deleted docs, gauges for currently running merges and their size.
+* shard replication and transaction log replay on replicas (TBD, SOLR-9856)
+* TBD: caches, update handler details, and other relevant SolrInfoMBean-s
+
+=== HTTP Registry (`solr.http`):
+
+* open / available / pending connections for shard handler and update handler
+
+=== Jetty Registry (`solr.jetty`):
+
+* threads and pools,
+* connection and request timers,
+* meters for responses by HTTP class (1xx, 2xx, etc)
+
+In the future, metrics will be added for shard leaders and cluster nodes, including aggregations from per-core metrics.
+
+== Reporters
+
+Reporter configurations are specified in `solr.xml` file in `<metrics><reporter>` sections, for example:
+
+[source,xml]
+----
+<solr>
+ <metrics>
+  <reporter name="graphite" group="node, jvm" class="org.apache.solr.metrics.reporters.SolrGraphiteReporter">
+    <str name="host">graphite-server</str>
+    <int name="port">9999</int>
+    <int name="period">60</int>
+  </reporter>
+  <reporter name="collection1Updates" registry="solr.core.collection1" class="org.apache.solr.metrics.reporters.SolrSlf4jReporter">
+    <int name="period">300</int>
+    <str name="prefix">example</str>
+    <str name="logger">updatesLogger</str>
+    <str name="filter">QUERYHANDLER./update</str>
+  </reporter>
+ </metrics>
+...
+</solr>
+----
+
+=== Reporter Arguments
+
+Reporter plugins use the following arguments:
+
+* *name* - (required) unique name of the reporter plugin
+* *class* - (required) fully-qualified implementation class of the plugin, must extend `SolrMetricReporter`
+* *group* - (optional) one or more of the predefined groups (see above)
+* *registry* - (optional) one or more of valid fully-qualified registry names
+* If both `group` and `registry` attributes are specified only the `group` attribute is considered. If neither attribute is specified then the plugin will be used for all groups and registries. Multiple group or registry names can be specified, separated by comma and/or space.
+
+Additionally, several implementation-specific initialization arguments can be specified in nested elements. There are some arguments that are common to SLF4J, Ganglia and Graphite reporters:
+
+* *period* - (optional int) period in seconds between reports. Default value is 60.
+* *prefix* - (optional str) prefix to be added to metric names, may be helpful in logical grouping of related Solr instances, e.g., machine name or cluster name. Default is empty string, ie. just the registry name and metric name will be used to form a fully-qualified metric name.
+* *filter* - (optional str) if not empty then only metric names that start with this value will be reported. Default is no filtering, ie. all metrics from selected registry will be reported.
+
+Reporters are instantiated for every group and registry that they were configured for, at the time when the respective components are initialized (e.g., on JVM startup or SolrCore load).
+
+When reporters are created their configuration is validated (and e.g., necessary connections are established). Uncaught errors at this initialization stage cause the reporter to be discarded from the running configuration.
+
+Reporters are closed when the corresponding component is being closed (e.g., on SolrCore close, or JVM shutdown) but metrics that they reported are still maintained in respective registries, as explained in the previous section.
+
+The following sections provide information on implementation-specific arguments. All implementation classes provided with Solr can be found under `org.apache.solr.metrics.reporters`.
+
+=== JMX Reporter
+
+The JMX Reporter uses the `org.apache.solr.metrics.reporters.SolrJmxReporter` class.
+
+It takes the following arguments:
+
+* *domain* - (optional str) JMX domain name. If not specified then registry name will be used.
+* *serviceUrl* - (optional str) service URL for a JMX server. If not specified then the default platform MBean server will be used.
+* *agentId* - (optional str) agent ID for a JMX server. Note: either `serviceUrl` or `agentId` can be specified but not both - if both are specified then the default MBean server will be used.
+
+Object names created by this reporter are hierarchical, dot-separated but also properly structured to form corresponding hierarchies in e.g., JConsole. This hierarchy consists of the following elements in the top-down order:
+
+* registry name (e.g., `solr.core.collection1.shard1.replica1`. Dot-separated registry names are also split into ObjectName hierarchy levels, so that metrics for this registry will be shown under `/solr/core/collection1/shard1/replica1` in JConsole, with each domain part being assigned to `dom1, dom2, ... domN` property.
+* reporter name (the value of reporter's `name` attribute)
+* category, scope and name for request handlers
+* or additional `name1, name2, ... nameN` elements for metrics from other components.
+
+=== SLF4J Reporter
+
+The SLF4J Reporter uses the `org.apache.solr.metrics.reporters.SolrSlf4jReporter` class.
+
+It takes the following arguments, in addition to the common arguments <<Reporter Arguments,above>>.
+
+* *logger* - (optional str) name of the logger to use. Default is empty, in which case the group or registry name will be used if specified in the plugin configuration.
+
+Users can specify logger name (and the corresponding logger configuration in e.g., Log4j configuration) to output metrics-related logging to separate file(s), which can then be processed by external applications.
+
+Each log line produced by this reporter consists of configuration-specific fields, and a message that follows this format:
+
+[source,text]
+----
+type=COUNTER, name={}, count={}
+
+type=GAUGE, name={}, value={}
+
+type=TIMER, name={}, count={}, min={}, max={}, mean={}, stddev={}, median={}, p75={}, p95={}, p98={}, p99={}, p999={}, mean_rate={}, m1={}, m5={}, m15={}, rate_unit={}, duration_unit={}
+
+type=METER, name={}, count={}, mean_rate={}, m1={}, m5={}, m15={}, rate_unit={}
+
+type=HISTOGRAM, name={}, count={}, min={}, max={}, mean={}, stddev={}, median={}, p75={}, p95={}, p98={}, p99={}, p999={}
+----
+
+(curly braces added only as placeholders for actual values).
+
+=== Graphite Reporter
+
+The http://graphiteapp.org[Graphite] Reporter uses the `org.apache.solr.metrics.reporters.SolrGraphiteReporter`) class.
+
+It takes the following attributes, in addition to the common attributes <<Reporter Arguments,above>>.
+
+* *host* - (required str) host name where Graphite server is running.
+* *port* - (required int) port number for the server
+* *pickled* - (optional bool) use "pickled" Graphite protocol which may be more efficient. Default is false (use plain-text protocol).
+
+When plain-text protocol is used (`pickled==false`) it's possible to use this reporter to integrate with systems other than Graphite, if they can accept space-separated and line-oriented input over network in the following format:
+
+[source,text]
+----
+dot.separated.metric.name[.and.attribute] value epochTimestamp
+----
+
+For example:
+
+[source,plain]
+----
+example.solr.node.cores.lazy 0 1482932097
+example.solr.node.cores.loaded 1 1482932097
+example.solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.2xx-responses.count 21 1482932097
+example.solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.2xx-responses.m1_rate 2.5474287707930614 1482932097
+example.solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.2xx-responses.m5_rate 3.8003171557510305 1482932097
+example.solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.2xx-responses.m15_rate 4.0623076220244245 1482932097
+example.solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.2xx-responses.mean_rate 0.5698031798408144 1482932097
+----
+
+=== Ganglia Reporter
+
+The http://ganglia.info[Ganglia] reporter uses the `org.apache.solr.metrics.reporters.SolrGangliaReporter` class.
+
+It take the following arguments, in addition to the common arguments <<Reporter Arguments,above>>.
+
+* *host* - (required str) host name where Ganglia server is running.
+* *port* - (required int) port number for the server
+* *multicast* - (optional bool) when true use multicast UDP communication, otherwise use UDP unicast. Default is false.
+
+== Core Level Metrics
+
+These metrics are available only on a per-core basis. Metrics that are aggregated across cores are not yet available.
+
+=== Index Merge Metrics
+
+These metrics are collected in respective registries for each core (e.g., `solr.core.collection1....`), under the `INDEX` category.
+
+Basic metrics are always collected - collection of additional metrics can be turned on using boolean parameters in the `/config/indexConfig/metrics` section of `solrconfig.xml`:
+
+[source,xml]
+----
+<config>
+  ...
+  <indexConfig>
+    <metrics>
+      <majorMergeDocs>524288</majorMergeDocs>
+      <bool name="mergeDetails">true</bool>
+      <bool name="directoryDetails">true</bool>
+    </metrics>
+    ...
+  </indexConfig>
+...
+</config>
+----
+
+The following metrics are collected:
+
+* `INDEX.merge.major` - timer for merge operations that include at least "majorMergeDocs" (default value for this parameter is 512k documents).
+* `INDEX.merge.minor` - timer for merge operations that include less than "majorMergeDocs".
+* `INDEX.merge.errors` - counter for merge errors.
+* `INDEX.flush` - meter for index flush operations.
+
+Additionally, the following gauges are reported, which help to monitor the momentary state of index merge operations:
+
+* `INDEX.merge.major.running` - number of running major merge operations (depending on the implementation of `MergeScheduler` that is used there can be several concurrently running merge operations).
+* `INDEX.merge.minor.running` - as above, for minor merge operations.
+* `INDEX.merge.major.running.docs` - total number of documents in the segments being currently merged in major merge operations.
+* `INDEX.merge.minor.running.docs` - as above, for minor merge operations.
+* `INDEX.merge.major.running.segments` - number of segments being currently merged in major merge operations.
+* `INDEX.merge.minor.running.segments` - as above, for minor merge operations.
+
+If the boolean flag `mergeDetails` is true then the following additional metrics are collected:
+
+* `INDEX.merge.major.docs` - meter for the number of documents merged in major merge operations
+* `INDEX.merge.major.deletedDocs` - meter for the number of deleted documents expunged in major merge operations
+
+== Metrics API
+
+The `admin/metrics` endpoint provides access to all the metrics for all metric groups.
+
+A few query parameters are available to limit the request:
+
+* *group*: The metric group to retrieve. The default is `all` to retrieve all metrics for all groups. Other possible values are: `jvm`, `jetty`, `node`, and `core`. More than one group can be specified in a request; multiple group names should be separated by a comma.
+* *type*: The type of metric to retrieve. The default is `all` to retrieve all metric types. Other possible values are `counter`, `gauge`, `histogram`, `meter`, and `timer`. More than one type can be specified in a request; multiple types should be separated by a comma.
+* *prefix*: The first characters of metric name that will filter the metrics returned to those starting with the provided string. It can be combined with group and/or type parameters. More than one prefix can be specified in a request; multiple prefixes should be separated by a comma. Prefix matching is also case-sensitive.
+
+Like other request handlers, the Metrics API can also take the `wt` parameter to define the output format.
+
+[[metrics_examples]]
+=== Examples
+
+Request only "counter" type metrics in the "core" group, returned in JSON:
+
+`\http://localhost:8983/solr/admin/metrics?wt=json&type=counter&group=core`
+
+Request only "core" group metrics that start with "INDEX", returned in JSON:
+
+`\http://localhost:8983/solr/admin/metrics?wt=json&prefix=INDEX&group=core`

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/morelikethis.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/morelikethis.adoc b/solr/solr-ref-guide/src/morelikethis.adoc
new file mode 100644
index 0000000..a99cc03
--- /dev/null
+++ b/solr/solr-ref-guide/src/morelikethis.adoc
@@ -0,0 +1,83 @@
+= MoreLikeThis
+:page-shortname: morelikethis
+:page-permalink: morelikethis.html
+
+The `MoreLikeThis` search component enables users to query for documents similar to a document in their result list.
+
+It does this by using terms from the original document to find similar documents in the index.
+
+There are three ways to use MoreLikeThis. The first, and most common, is to use it as a request handler. In this case, you would send text to the MoreLikeThis request handler as needed (as in when a user clicked on a "similar documents" link).
+
+The second is to use it as a search component. This is less desirable since it performs the MoreLikeThis analysis on every document returned. This may slow search results.
+
+The final approach is to use it as a request handler but with externally supplied text. This case, also referred to as the MoreLikeThisHandler, will supply information about similar documents in the index based on the text of the input document.
+
+[[MoreLikeThis-HowMoreLikeThisWorks]]
+== How MoreLikeThis Works
+
+`MoreLikeThis` constructs a Lucene query based on terms in a document. It does this by pulling terms from the defined list of fields ( see the `mlt.fl` parameter, below). For best results, the fields should have stored term vectors in `schema.xml`. For example:
+
+[source,xml]
+----
+<field name="cat" ... termVectors="true" />
+----
+
+If term vectors are not stored, `MoreLikeThis` will generate terms from stored fields. A `uniqueKey` must also be stored in order for MoreLikeThis to work properly.
+
+The next phase filters terms from the original document using thresholds defined with the MoreLikeThis parameters. Finally, a query is run with these terms, and any other query parameters that have been defined (see the `mlt.qf` parameter, below) and a new document set is returned.
+
+[[MoreLikeThis-CommonParametersforMoreLikeThis]]
+== Common Parameters for MoreLikeThis
+
+The table below summarizes the `MoreLikeThis` parameters supported by Lucene/Solr. These parameters can be used with any of the three possible MoreLikeThis approaches.
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|mlt.fl |Specifies the fields to use for similarity. If possible, these should have stored `termVectors`.
+|mlt.mintf |Specifies the Minimum Term Frequency, the frequency below which terms will be ignored in the source document.
+|mlt.mindf |Specifies the Minimum Document Frequency, the frequency at which words will be ignored which do not occur in at least this many documents.
+|mlt.maxdf |Specifies the Maximum Document Frequency, the frequency at which words will be ignored which occur in more than this many documents.
+|mlt.minwl |Sets the minimum word length below which words will be ignored.
+|mlt.maxwl |Sets the maximum word length above which words will be ignored.
+|mlt.maxqt |Sets the maximum number of query terms that will be included in any generated query.
+|mlt.maxntp |Sets the maximum number of tokens to parse in each example document field that is not stored with TermVector support.
+|mlt.boost |Specifies if the query will be boosted by the interesting term relevance. It can be either "true" or "false".
+|mlt.qf |Query fields and their boosts using the same format as that used by the <<the-dismax-query-parser.adoc#the-dismax-query-parser,DisMax Query Parser>>. These fields must also be specified in `mlt.fl`.
+|===
+
+[[MoreLikeThis-ParametersfortheMoreLikeThisComponent]]
+== Parameters for the MoreLikeThisComponent
+
+Using MoreLikeThis as a search component returns similar documents for each document in the response set. In addition to the common parameters, these additional options are available:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|mlt |If set to true, activates the `MoreLikeThis` component and enables Solr to return `MoreLikeThis` results.
+|mlt.count |Specifies the number of similar documents to be returned for each result. The default value is 5.
+|===
+
+[[MoreLikeThis-ParametersfortheMoreLikeThisHandler]]
+== Parameters for the MoreLikeThisHandler
+
+The table below summarizes parameters accessible through the `MoreLikeThisHandler`. It supports faceting, paging, and filtering using common query parameters, but does not work well with alternate query parsers.
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|mlt.match.include |Specifies whether or not the response should include the matched document. If set to false, the response will look like a normal select response.
+|mlt.match.offset |Specifies an offset into the main query search results to locate the document on which the `MoreLikeThis` query should operate. By default, the query operates on the first result for the q parameter.
+|mlt.interestingTerms |Controls how the `MoreLikeThis` component presents the "interesting" terms (the top TF/IDF terms) for the query. Supports three settings. The setting list lists the terms. The setting none lists no terms. The setting details lists the terms along with the boost value used for each term. Unless `mlt.boost=true`, all terms will have `boost=1.0`.
+|===
+
+[[MoreLikeThis-MoreLikeThisQueryParser]]
+== More Like This Query Parser
+
+The `mlt` query parser provides a mechanism to retrieve documents similar to a given document, like the handler. More information on the usage of the mlt query parser can be found int the section <<other-parsers.adoc#other-parsers,Other Parsers>>.

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/near-real-time-searching.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/near-real-time-searching.adoc b/solr/solr-ref-guide/src/near-real-time-searching.adoc
new file mode 100644
index 0000000..f4396f3
--- /dev/null
+++ b/solr/solr-ref-guide/src/near-real-time-searching.adoc
@@ -0,0 +1,121 @@
+= Near Real Time Searching
+:page-shortname: near-real-time-searching
+:page-permalink: near-real-time-searching.html
+
+Near Real Time (NRT) search means that documents are available for search almost immediately after being indexed.
+
+This allows additions and updates to documents to be seen in 'near' real time. Solr does not block updates while a commit is in progress. Nor does it wait for background merges to complete before opening a new search of indexes and returning.
+
+With NRT, you can modify a `commit` command to be a *soft commit*, which avoids parts of a standard commit that can be costly. You will still want to do standard commits to ensure that documents are in stable storage, but *soft commits* let you see a very near real time view of the index in the meantime.
+
+However, pay special attention to cache and autowarm settings as they can have a significant impact on NRT performance.
+
+[[NearRealTimeSearching-CommitsandOptimizing]]
+== Commits and Optimizing
+
+A commit operation makes index changes visible to new search requests. A *hard commit* uses the transaction log to get the id of the latest document changes, and also calls `fsync` on the index files to ensure they have been flushed to stable storage and no data loss will result from a power failure.
+
+A *soft commit* is much faster since it only makes index changes visible and does not `fsync` index files or write a new index descriptor. If the JVM crashes or there is a loss of power, changes that occurred after the last *hard commit* will be lost. Search collections that have NRT requirements (that want index changes to be quickly visible to searches) will want to soft commit often but hard commit less frequently. A softCommit may be "less expensive" in terms of time, but not free, since it can slow throughput.
+
+An *optimize* is like a *hard commit* except that it forces all of the index segments to be merged into a single segment first. Depending on the use, this operation should be performed infrequently (e.g., nightly), if at all, since it involves reading and re-writing the entire index. Segments are normally merged over time anyway (as determined by the merge policy), and optimize just forces these merges to occur immediately.
+
+Soft commit takes uses two parameters: `maxDocs` and `maxTime`.
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|`maxDocs` |Integer. Defines the number of documents to queue before pushing them to the index. It works in conjunction with the `update_handler_autosoftcommit_max_time` parameter in that if either limit is reached, the documents will be pushed to the index.
+|`maxTime` |The number of milliseconds to wait before pushing documents to the index. It works in conjunction with the `update_handler_autosoftcommit_max_docs` parameter in that if either limit is reached, the documents will be pushed to the index.
+|===
+
+Use `maxDocs` and `maxTime` judiciously to fine-tune your commit strategies.
+
+[[NearRealTimeSearching-AutoCommits]]
+=== AutoCommits
+
+An autocommit also uses the parameters `maxDocs` and `maxTime`. However it's useful in many strategies to use both a hard `autocommit` and `autosoftcommit` to achieve more flexible commits.
+
+A common configuration is to do a hard `autocommit` every 1-10 minutes and a `autosoftcommit` every second. With this configuration, new documents will show up within about a second of being added, and if the power goes out, soft commits are lost unless a hard commit has been done.
+
+For example:
+
+[source,xml]
+----
+<autoSoftCommit>
+  <maxTime>1000</maxTime>
+</autoSoftCommit>
+----
+
+It's better to use `maxTime` rather than `maxDocs` to modify an `autoSoftCommit`, especially when indexing a large number of documents through the commit operation. It's also better to turn off `autoSoftCommit` for bulk indexing.
+
+[[NearRealTimeSearching-OptionalAttributesforcommitandoptimize]]
+=== Optional Attributes for `commit` and `optimize`
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="20,20,60",options="header"]
+|===
+|Parameter |Valid Attributes |Description
+|`waitSearcher` |true, false |Block until a new searcher is opened and registered as the main query searcher, making the changes visible. Default is true.
+|`softCommit` |true, false |Perform a soft commit. This will refresh the view of the index faster, but without guarantees that the document is stably stored. Default is false.
+|`expungeDeletes` |true, false |Valid for `commit` only. This parameter purges deleted data from segments. The default is false.
+|`maxSegments` |integer |Valid for `optimize` only. Optimize down to at most this number of segments. The default is 1.
+|===
+
+Example of `commit` and `optimize` with optional attributes:
+
+[source,xml]
+----
+<commit waitSearcher="false"/>
+<commit waitSearcher="false" expungeDeletes="true"/>
+<optimize waitSearcher="false"/>
+----
+
+[[NearRealTimeSearching-PassingcommitandcommitWithinparametersaspartoftheURL]]
+=== Passing `commit` and `commitWithin` Parameters as Part of the URL
+
+Update handlers can also get `commit`-related parameters as part of the update URL. This example adds a small test document and causes an explicit commit to happen immediately afterwards:
+
+[source,text]
+----
+http://localhost:8983/solr/my_collection/update?stream.body=<add><doc>
+   <field name="id">testdoc</field></doc></add>&commit=true
+----
+
+Alternately, you may want to use this:
+
+[source,text]
+----
+http://localhost:8983/solr/my_collection/update?stream.body=<optimize/>
+----
+
+This example causes the index to be optimized down to at most 10 segments, but won't wait around until it's done (`waitFlush=false`):
+
+[source,bash]
+----
+curl 'http://localhost:8983/solr/my_collection/update?optimize=true&maxSegments=10&waitFlush=false'
+----
+
+This example adds a small test document with a `commitWithin` instruction that tells Solr to make sure the document is committed no later than 10 seconds later (this method is generally preferred over explicit commits):
+
+[source,bash]
+----
+curl http://localhost:8983/solr/my_collection/update?commitWithin=10000
+  -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">testdoc</field></doc></add>'
+----
+
+[[NearRealTimeSearching-ChangingdefaultcommitWithinBehavior]]
+=== Changing default `commitWithin` Behavior
+
+The `commitWithin` settings allow forcing document commits to happen in a defined time period. This is used most frequently with <<near-real-time-searching.adoc#near-real-time-searching,Near Real Time Searching>>, and for that reason the default is to perform a soft commit. This does not, however, replicate new documents to slave servers in a master/slave environment. If that's a requirement for your implementation, you can force a hard commit by adding a parameter, as in this example:
+
+[source,xml]
+----
+<commitWithin>
+  <softCommit>false</softCommit>
+</commitWithin>
+----
+
+With this configuration, when you call `commitWithin` as part of your update message, it will automatically perform a hard commit every time.

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/other-parsers.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/other-parsers.adoc b/solr/solr-ref-guide/src/other-parsers.adoc
new file mode 100644
index 0000000..290c5b8
--- /dev/null
+++ b/solr/solr-ref-guide/src/other-parsers.adoc
@@ -0,0 +1,978 @@
+= Other Parsers
+:page-shortname: other-parsers
+:page-permalink: other-parsers.html
+
+In addition to the main query parsers discussed earlier, there are several other query parsers that can be used instead of or in conjunction with the main parsers for specific purposes.
+
+This section details the other parsers, and gives examples for how they might be used.
+
+Many of these parsers are expressed the same way as <<local-parameters-in-queries.adoc#local-parameters-in-queries,Local Parameters in Queries>>.
+
+[[OtherParsers-BlockJoinQueryParsers]]
+== Block Join Query Parsers
+
+There are two query parsers that support block joins. These parsers allow indexing and searching for relational content that has been<<uploading-data-with-index-handlers.adoc#uploading-data-with-index-handlers,indexed as nested documents>>.
+
+The example usage of the query parsers below assumes these two documents and each of their child documents have been indexed:
+
+[source,xml]
+----
+<add>
+  <doc>
+  <field name="id">1</field>
+  <field name="title">Solr has block join support</field>
+  <field name="content_type">parentDocument</field>
+    <doc>
+      <field name="id">2</field>
+      <field name="comments">SolrCloud supports it too!</field>
+    </doc>
+  </doc>
+  <doc>
+    <field name="id">3</field>
+    <field name="title">New Lucene and Solr release</field>
+    <field name="content_type">parentDocument</field>
+    <doc>
+      <field name="id">4</field>
+      <field name="comments">Lots of new features</field>
+    </doc>
+  </doc>
+</add>
+----
+
+[[OtherParsers-BlockJoinChildrenQueryParser]]
+=== Block Join Children Query Parser
+
+This parser takes a query that matches some parent documents and returns their children.
+
+The syntax for this parser is: `q={!child of=<allParents>}<someParents>`.
+
+The parameter `allParents` is a filter that matches *only parent documents*; here you would define the field and value that you used to identify *all parent documents*.
+
+The parameter `someParents` identifies a query that will match some of the parent documents. The output is the children.
+
+Using the example documents above, we can construct a query such as `q={!child of="content_type:parentDocument"}title:lucene`. We only get one document in response:
+
+[source,xml]
+----
+<result name="response" numFound="1" start="0">
+  <doc>
+    <str name="id">4</str>
+    <str name="comments">Lots of new features</str>
+  </doc>
+</result>
+----
+
+Note that the query for `someParents` should match only parent documents passed by `allParents` or you may get an exception:
+
+....
+Parent query must not match any docs besides parent filter. Combine them as must (+) and must-not (-) clauses to find a problem doc.
+....
+In older version the error is:
+....
+Parent query yields document which is not matched by parents filter.
+....
+You can search for `q=+(someParents) -(allParents)` to find a cause.
+
+[[OtherParsers-BlockJoinParentQueryParser]]
+=== Block Join Parent Query Parser
+
+This parser takes a query that matches child documents and returns their parents.
+
+The syntax for this parser is similar: `q={!parent which=<allParents>}<someChildren>`.
+
+The parameter `allParents` is a filter that matches *only parent documents*; here you would define the field and value that you used to identify *all parent documents*.
+
+The parameter `someChildren` is a query that matches some or all of the child documents.
+
+Note that the query for `someChildren` should match only child documents or you may get an exception:
+....
+Child query must not match same docs with parent filter. Combine them as must clauses (+) to find a problem doc.
+....
+In older version it's:
+....
+child query must only match non-parent docs.
+....
+You can search for `q=+(parentFilter) +(someChildren)` to find a cause .
+
+Again using the example documents above, we can construct a query such as `q={!parent which="content_type:parentDocument"}comments:SolrCloud`. We get this document in response:
+
+[source,xml]
+----
+<result name="response" numFound="1" start="0">
+  <doc>
+    <str name="id">1</str>
+    <arr name="title"><str>Solr has block join support</str></arr>
+    <arr name="content_type"><str>parentDocument</str></arr>
+  </doc>
+</result>
+----
+
+.Using which
+[WARNING]
+====
+A common mistake is to try to filter parents with a `which` filter, as in this bad example:
+
+`q={!parent which="*title:join*"}comments:SolrCloud`
+
+Instead, you should use a sibling mandatory clause as a filter:
+
+`q= *+title:join* +{!parent which="*content_type:parentDocument*"}comments:SolrCloud`
+
+====
+
+[[OtherParsers-Scoring]]
+=== Scoring
+
+You can optionally use the `score` local parameter to return scores of the subordinate query. The values to use for this parameter define the type of aggregation, which are `avg` (average), `max` (maximum), `min` (minimum), `total (sum)`. Implicit default is `none` which returns `0.0`.
+
+[[OtherParsers-BoostQueryParser]]
+== Boost Query Parser
+
+`BoostQParser` extends the `QParserPlugin` and creates a boosted query from the input value. The main value is the query to be boosted. Parameter `b` is the function query to use as the boost. The query to be boosted may be of any type.
+
+Examples:
+
+Creates a query "foo" which is boosted (scores are multiplied) by the function query `log(popularity)`:
+
+[source,text]
+----
+{!boost b=log(popularity)}foo
+----
+
+Creates a query "foo" which is boosted by the date boosting function referenced in `ReciprocalFloatFunction`:
+
+[source,text]
+----
+{!boost b=recip(ms(NOW,mydatefield),3.16e-11,1,1)}foo
+----
+
+[[OtherParsers-CollapsingQueryParser]]
+== Collapsing Query Parser
+
+The `CollapsingQParser` is really a _post filter_ that provides more performant field collapsing than Solr's standard approach when the number of distinct groups in the result set is high.
+
+This parser collapses the result set to a single document per group before it forwards the result set to the rest of the search components. So all downstream components (faceting, highlighting, etc.) will work with the collapsed result set.
+
+Details about using the `CollapsingQParser` can be found in the section <<collapse-and-expand-results.adoc#collapse-and-expand-results,Collapse and Expand Results>>.
+
+[[OtherParsers-ComplexPhraseQueryParser]]
+== Complex Phrase Query Parser
+
+The `ComplexPhraseQParser` provides support for wildcards, ORs, etc., inside phrase queries using Lucene's {lucene-javadocs}/queryparser/org/apache/lucene/queryparser/complexPhrase/ComplexPhraseQueryParser.html[`ComplexPhraseQueryParser`].
+
+Under the covers, this query parser makes use of the Span group of queries, e.g., spanNear, spanOr, etc., and is subject to the same limitations as that family or parsers.
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|`inOrder` |Set to true to force phrase queries to match terms in the order specified. Default: *true*
+|`df` |The default search field.
+|===
+
+*Examples:*
+
+[source,text]
+----
+{!complexphrase inOrder=true}name:"Jo* Smith"
+----
+
+[source,text]
+----
+{!complexphrase inOrder=false}name:"(john jon jonathan~) peters*"
+----
+
+A mix of ordered and unordered complex phrase queries:
+
+[source,text]
+----
++_query_:"{!complexphrase inOrder=true}manu:\"a* c*\"" +_query_:"{!complexphrase inOrder=false df=name}\"bla* pla*\""
+----
+
+[[OtherParsers-Limitations]]
+=== Limitations
+
+Performance is sensitive to the number of unique terms that are associated with a pattern. For instance, searching for "a*" will form a large OR clause (technically a SpanOr with many terms) for all of the terms in your index for the indicated field that start with the single letter 'a'. It may be prudent to restrict wildcards to at least two or preferably three letters as a prefix. Allowing very short prefixes may result in to many low-quality documents being returned.
+
+Notice that it also supports leading wildcards "*a" as well with consequent performance implications. Applying <<filter-descriptions.adoc#reversed-wildcard-filter,ReversedWildcardFilterFactory>> in index-time analysis is usually a good idea.
+
+[[OtherParsers-MaxBooleanClauses]]
+==== MaxBooleanClauses
+
+You may need to increase MaxBooleanClauses in `solrconfig.xml` as a result of the term expansion above:
+
+[source,xml]
+----
+<maxBooleanClauses>4096</maxBooleanClauses>
+----
+
+This property is described in more detail in the section <<query-settings-in-solrconfig.adoc#QuerySettingsinSolrConfig-QuerySizingandWarming,Query Sizing and Warming>>.
+
+[[OtherParsers-Stopwords]]
+==== Stopwords
+
+It is recommended not to use stopword elimination with this query parser.
+
+Lets say we add the terms *the*, *up*, and *to* to `stopwords.txt` for your collection, and index a document containing the text _"Stores up to 15,000 songs, 25,00 photos, or 150 yours of video"_ in a field named "features".
+
+While the query below does not use this parser:
+
+[source,text]
+----
+ q=features:"Stores up to 15,000"
+----
+
+the document is returned. The next query that _does_ use the Complex Phrase Query Parser, as in this query:
+
+[source,text]
+----
+ q=features:"sto* up to 15*"&defType=complexphrase
+----
+
+does _not_ return that document because SpanNearQuery has no good way to handle stopwords in a way analogous to PhraseQuery. If you must remove stopwords for your use case, use a custom filter factory or perhaps a customized synonyms filter that reduces given stopwords to some impossible token.
+
+[[OtherParsers-Escaping]]
+==== Escaping
+
+Special care has to be given when escaping: clauses between double quotes (usually whole query) is parsed twice, these parts have to be escaped as twice. eg `"foo\\: bar\\^"`.
+
+[[OtherParsers-FieldQueryParser]]
+== Field Query Parser
+
+The `FieldQParser` extends the `QParserPlugin` and creates a field query from the input value, applying text analysis and constructing a phrase query if appropriate. The parameter `f` is the field to be queried.
+
+Example:
+
+[source,text]
+----
+{!field f=myfield}Foo Bar
+----
+
+This example creates a phrase query with "foo" followed by "bar" (assuming the analyzer for `myfield` is a text field with an analyzer that splits on whitespace and lowercase terms). This is generally equivalent to the Lucene query parser expression `myfield:"Foo Bar"`.
+
+[[OtherParsers-FunctionQueryParser]]
+== Function Query Parser
+
+The `FunctionQParser` extends the `QParserPlugin` and creates a function query from the input value. This is only one way to use function queries in Solr; for another, more integrated, approach, see the section on <<function-queries.adoc#function-queries,Function Queries>>.
+
+Example:
+
+[source,text]
+----
+{!func}log(foo)
+----
+
+[[OtherParsers-FunctionRangeQueryParser]]
+== Function Range Query Parser
+
+The `FunctionRangeQParser` extends the `QParserPlugin` and creates a range query over a function. This is also referred to as `frange`, as seen in the examples below.
+
+Other parameters:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|l |The lower bound, optional
+|u |The upper bound, optional
+|incl |Include the lower bound: true/false, optional, default=true
+|incu |Include the upper bound: true/false, optional, default=true
+|===
+
+Examples:
+
+[source,text]
+----
+{!frange l=1000 u=50000}myfield
+----
+
+[source,text]
+----
+ fq={!frange l=0 u=2.2} sum(user_ranking,editor_ranking)
+----
+
+Both of these examples restrict the results by a range of values found in a declared field or a function query. In the second example, we're doing a sum calculation, and then defining only values between 0 and 2.2 should be returned to the user.
+
+For more information about range queries over functions, see Yonik Seeley's introductory blog post https://lucidworks.com/2009/07/06/ranges-over-functions-in-solr-14/[Ranges over Functions in Solr 1.4].
+
+[[OtherParsers-GraphQueryParser]]
+== Graph Query Parser
+
+The `graph` query parser does a breadth first, cyclic aware, graph traversal of all documents that are "reachable" from a starting set of root documents identified by a wrapped query.
+
+The graph is built according to linkages between documents based on the terms found in "```from```" and "```to```" fields that you specify as part of the query
+
+[[OtherParsers-Parameters]]
+=== Parameters
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|to |The field name of matching documents to inspect to identify outgoing edges for graph traversal. Defaults to `edge_ids` .
+|from |The field name to of candidate documents to inspect to identify incoming graph edges. Defaults to `node_id` .
+|traversalFilter |An optional query that can be supplied to limit the scope of documents that are traversed.
+|maxDepth |Integer specifying how deep the breadth first search of the graph should go begining with the initial query. Defaults to -1 (unlimited)
+|returnRoot |Boolean to indicate if the documents that matched the original query (to define the starting points for graph) should be included in the final results. Defaults to true
+|returnOnlyLeaf |Boolean that indicates if the results of the query should be filtered so that only documents with no outgoing edges are returned. Defaults to false
+|useAutn |Boolean that indicates if an Automatons should be compiled for each iteration of the breadth first search, which may be faster for some graphs. Defaults to false.
+|===
+
+[[OtherParsers-Limitations.1]]
+=== Limitations
+
+The `graph` parser only works in single node Solr installations, or with <<solrcloud.adoc#solrcloud,SolrCloud>> collections that use exactly 1 shard.
+
+[[OtherParsers-Examples]]
+=== Examples
+
+To understand how the graph parser works, consider the following Directed Cyclic Graph, containing 8 nodes (A to H) and 9 edges (1 to 9):
+
+image::images/other-parsers/graph_qparser_example.png[image,height=200]
+
+One way to model this graph as Solr documents, would be to create one document per node, with mutivalued fields identifying the incoming and outgoing edges for each node:
+
+[source,bash]
+----
+curl -H 'Content-Type: application/json' 'http://localhost:8983/solr/my_graph/update?commit=true' --data-binary '[
+  {"id":"A","foo":  7, "out_edge":["1","9"],  "in_edge":["4","2"]  },
+  {"id":"B","foo": 12, "out_edge":["3","6"],  "in_edge":["1"]      },
+  {"id":"C","foo": 10, "out_edge":["5","2"],  "in_edge":["9"]      },
+  {"id":"D","foo": 20, "out_edge":["4","7"],  "in_edge":["3","5"]  },
+  {"id":"E","foo": 17, "out_edge":[],         "in_edge":["6"]      },
+  {"id":"F","foo": 11, "out_edge":[],         "in_edge":["7"]      },
+  {"id":"G","foo":  7, "out_edge":["8"],      "in_edge":[]         },
+  {"id":"H","foo": 10, "out_edge":[],         "in_edge":["8"]      }
+]'
+----
+
+With the model shown above, the following query demonstrates a simple traversal of all nodes reachable from node A:
+
+[source,text]
+----
+http://localhost:8983/solr/my_graph/query?fl=id&q={!graph+from=in_edge+to=out_edge}id:A
+----
+
+[source,json]
+----
+"response":{"numFound":6,"start":0,"docs":[
+   { "id":"A" },
+   { "id":"B" },
+   { "id":"C" },
+   { "id":"D" },
+   { "id":"E" },
+   { "id":"F" } ]
+}
+----
+
+We can also use the `traversalFilter` to limit the graph traversal to only nodes with maximum value of 15 in the `foo` field. In this case that means D, E, and F are excluded – F has a value of `foo=11`, but it is unreachable because the traversal skipped D:
+
+[source,text]
+----
+http://localhost:8983/solr/my_graph/query?fl=id&q={!graph+from=in_edge+to=out_edge+traversalFilter='foo:[*+TO+15]'}id:A
+----
+
+[source,json]
+----
+...
+"response":{"numFound":3,"start":0,"docs":[
+   { "id":"A" },
+   { "id":"B" },
+   { "id":"C" } ]
+}
+----
+
+The examples shown so far have all used a query for a single document (`"id:A"`) as the root node for the graph traversal, but any query can be used to identify multiple documents to use as root nodes. The next example demonstrates using the `maxDepth` param to find all nodes that are at most one edge away from an root node with a value in the `foo` field less then or equal to 10:
+
+[source,text]
+----
+http://localhost:8983/solr/my_graph/query?fl=id&q={!graph+from=in_edge+to=out_edge+maxDepth=1}foo:[*+TO+10]
+----
+
+[source,json]
+----
+...
+"response":{"numFound":6,"start":0,"docs":[
+   { "id":"A" },
+   { "id":"B" },
+   { "id":"C" },
+   { "id":"D" },
+   { "id":"G" },
+   { "id":"H" } ]
+}
+----
+
+[[OtherParsers-SimplifiedModels]]
+=== Simplified Models
+
+The Document & Field modeling used in the above examples enumerated all of the outgoing and income edges for each node explicitly, to help demonstrate exactly how the "from" and "to" params work, and to give you an idea of what is possible. With multiple sets of fields like these for identifying incoming and outgoing edges, it's possible to model many independent Directed Graphs that contain some or all of the documents in your collection.
+
+But in many cases it can also be possible to drastically simplify the model used.
+
+For example, the same graph shown in the diagram above can be modelled by Solr Documents that represent each node and know only the ids of the nodes they link to, with out knowing anything about the incoming links:
+
+[source,bash]
+----
+curl -H 'Content-Type: application/json' 'http://localhost:8983/solr/alt_graph/update?commit=true' --data-binary '[
+  {"id":"A","foo":  7, "out_edge":["B","C"] },
+  {"id":"B","foo": 12, "out_edge":["E","D"] },
+  {"id":"C","foo": 10, "out_edge":["A","D"] },
+  {"id":"D","foo": 20, "out_edge":["A","F"] },
+  {"id":"E","foo": 17, "out_edge":[]        },
+  {"id":"F","foo": 11, "out_edge":[]        },
+  {"id":"G","foo":  7, "out_edge":["H"]     },
+  {"id":"H","foo": 10, "out_edge":[]        }
+  ]'
+----
+
+With this alternative document model, all of the same queries demonstrated above can still be executed, simply by changing the "```from```" param to replace the "```in_edge```" field with the "```id```" field:
+
+[source,text]
+----
+http://localhost:8983/solr/alt_graph/query?fl=id&q={!graph+from=id+to=out_edge+maxDepth=1}foo:[*+TO+10]
+----
+
+[source,json]
+----
+...
+"response":{"numFound":6,"start":0,"docs":[
+   { "id":"A" },
+   { "id":"B" },
+   { "id":"C" },
+   { "id":"D" },
+   { "id":"G" },
+   { "id":"H" } ]
+}
+----
+
+[[OtherParsers-JoinQueryParser]]
+== Join Query Parser
+
+`JoinQParser` extends the `QParserPlugin`. It allows normalizing relationships between documents with a join operation. This is different from the concept of a join in a relational database because no information is being truly joined. An appropriate SQL analogy would be an "inner query".
+
+Examples:
+
+Find all products containing the word "ipod", join them against manufacturer docs and return the list of manufacturers:
+
+[source,text]
+----
+{!join from=manu_id_s to=id}ipod
+----
+
+Find all manufacturer docs named "belkin", join them against product docs, and filter the list to only products with a price less than $12:
+
+[source,text]
+----
+q  = {!join from=id to=manu_id_s}compName_s:Belkin
+fq = price:[* TO 12]
+----
+
+The join operation is done on a term basis, so the "from" and "to" fields must use compatible field types. For example: joining between a `StrField` and a `TrieIntField` will not work, likewise joining between a `StrField` and a `TextField` that uses `LowerCaseFilterFactory` will only work for values that are already lower cased in the string field.
+
+[[OtherParsers-Scoring.1]]
+=== Scoring
+
+You can optionally use the `score` parameter to return scores of the subordinate query. The values to use for this parameter define the type of aggregation, which are `avg` (average), `max` (maximum), `min` (minimum) `total`, or `none`.
+
+.Score parameter and single value numerics
+[WARNING]
+====
+Specifying `score` local parameter switches the join algorithm. This might have performance implication on large indices, but it's more important that this algorithm won't work for single value numeric field starting from 7.0. Users are encouraged to change field types to string and rebuild indexes during migration.
+====
+
+[[OtherParsers-JoiningAcrossCollections]]
+=== Joining Across Collections
+
+You can also specify a `fromIndex` parameter to join with a field from another core or collection. If running in SolrCloud mode, then the collection specified in the `fromIndex` parameter must have a single shard and a replica on all Solr nodes where the collection you're joining to has a replica.
+
+Let's consider an example where you want to use a Solr join query to filter movies by directors that have won an Oscar. Specifically, imagine we have two collections with the following fields:
+
+*movies*: id, title, director_id, ...
+
+*movie_directors*: id, name, has_oscar, ...
+
+To filter movies by directors that have won an Oscar using a Solr join on the *movie_directors* collection, you can send the following filter query to the *movies* collection:
+
+[source,text]
+----
+fq={!join from=id fromIndex=movie_directors to=director_id}has_oscar:true
+----
+
+Notice that the query criteria of the filter (`has_oscar:true`) is based on a field in the collection specified using `fromIndex`. Keep in mind that you cannot return fields from the `fromIndex` collection using join queries, you can only use the fields for filtering results in the "to" collection (movies).
+
+Next, let's understand how these collections need to be deployed in your cluster. Imagine the *movies* collection is deployed to a four node SolrCloud cluster and has two shards with a replication factor of two. Specifically, the *movies* collection has replicas on the following four nodes:
+
+node 1: movies_shard1_replica1
+
+node 2: movies_shard1_replica2
+
+node 3: movies_shard2_replica1
+
+node 4: movies_shard2_replica2
+
+To use the *movie_directors* collection in Solr join queries with the *movies* collection, it needs to have a replica on each of the four nodes. In other words, *movie_directors* must have one shard and replication factor of four:
+
+node 1: movie_directors_shard1_replica1
+
+node 2: movie_directors_shard1_replica2
+
+node 3: movie_directors_shard1_replica3
+
+node 4: movie_directors_shard1_replica4
+
+At query time, the `JoinQParser` will access the local replica of the *movie_directors* collection to perform the join. If a local replica is not available or active, then the query will fail. At this point, it should be clear that since you're limited to a single shard and the data must be replicated across all nodes where it is needed, this approach works better with smaller data sets where there is a one-to-many relationship between the from collection and the to collection. Moreover, if you add a replica to the to collection, then you also need to add a replica for the from collection.
+
+For more information about join queries, see the Solr Wiki page on http://wiki.apache.org/solr/Join[Joins]. Erick Erickson has also written a blog post about join performance titled https://lucidworks.com/2012/06/20/solr-and-joins/[Solr and Joins].
+
+[[OtherParsers-LuceneQueryParser]]
+== Lucene Query Parser
+
+The `LuceneQParser` extends the `QParserPlugin` by parsing Solr's variant on the Lucene QueryParser syntax. This is effectively the same query parser that is used in Lucene. It uses the operators `q.op`, the default operator ("OR" or "AND") and `df`, the default field name.
+
+Example:
+
+[source,text]
+----
+{!lucene q.op=AND df=text}myfield:foo +bar -baz
+----
+
+For more information about the syntax for the Lucene Query Parser, see the {lucene-javadocs}/queryparser/org/apache/lucene/queryparser/classic/package-summary.html[Classic QueryParser javadocs].
+
+[[OtherParsers-LearningToRankQueryParser]]
+== Learning To Rank Query Parser
+
+The `LTRQParserPlugin` is a special purpose parser for reranking the top results of a simple query using a more complex ranking query which is based on a machine learnt model.
+
+Example:
+
+[source,text]
+----
+{!ltr model=myModel reRankDocs=100}
+----
+
+Details about using the `LTRQParserPlugin` can be found in the <<learning-to-rank.adoc#learning-to-rank,Learning To Rank>> section.
+
+[[OtherParsers-MaxScoreQueryParser]]
+== Max Score Query Parser
+
+The `MaxScoreQParser` extends the `LuceneQParser` but returns the Max score from the clauses. It does this by wrapping all `SHOULD` clauses in a `DisjunctionMaxQuery` with tie=1.0. Any `MUST` or `PROHIBITED` clauses are passed through as-is. Non-boolean queries, e.g., NumericRange falls-through to the `LuceneQParser` parser behavior.
+
+Example:
+
+[source,text]
+----
+{!maxscore tie=0.01}C OR (D AND E)
+----
+
+[[OtherParsers-MoreLikeThisQueryParser]]
+== More Like This Query Parser
+
+`MLTQParser` enables retrieving documents that are similar to a given document. It uses Lucene's existing `MoreLikeThis` logic and also works in SolrCloud mode. The document identifier used here is the unique id value and not the Lucene internal document id. The list of returned documents excludes the queried document.
+
+This query parser takes the following parameters:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|qf |Specifies the fields to use for similarity.
+|mintf |Specifies the Minimum Term Frequency, the frequency below which terms will be ignored in the source document.
+|mindf |Specifies the Minimum Document Frequency, the frequency at which words will be ignored when they do not occur in at least this many documents.
+|maxdf |Specifies the Maximum Document Frequency, the frequency at which words will be ignored when they occur in more than this many documents.
+|minwl |Sets the minimum word length below which words will be ignored.
+|maxwl |Sets the maximum word length above which words will be ignored.
+|maxqt |Sets the maximum number of query terms that will be included in any generated query.
+|maxntp |Sets the maximum number of tokens to parse in each example document field that is not stored with TermVector support.
+|boost |Specifies if the query will be boosted by the interesting term relevance. It can be either "true" or "false".
+|===
+
+Examples:
+
+Find documents like the document with id=1 and using the `name` field for similarity.
+
+[source,text]
+----
+{!mlt qf=name}1
+----
+
+Adding more constraints to what qualifies as similar using mintf and mindf.
+
+[source,text]
+----
+{!mlt qf=name mintf=2 mindf=3}1
+----
+
+[[OtherParsers-NestedQueryParser]]
+== Nested Query Parser
+
+The `NestedParser` extends the `QParserPlugin` and creates a nested query, with the ability for that query to redefine its type via local parameters. This is useful in specifying defaults in configuration and letting clients indirectly reference them.
+
+Example:
+
+[source,text]
+----
+{!query defType=func v=$q1}
+----
+
+If the `q1` parameter is price, then the query would be a function query on the price field. If the `q1` parameter is \{!lucene}inStock:true}} then a term query is created from the Lucene syntax string that matches documents with `inStock=true`. These parameters would be defined in `solrconfig.xml`, in the `defaults` section:
+
+[source,xml]
+----
+<lst name="defaults">
+  <str name="q1">{!lucene}inStock:true</str>
+</lst>
+----
+
+For more information about the possibilities of nested queries, see Yonik Seeley's blog post https://lucidworks.com/2009/03/31/nested-queries-in-solr/[Nested Queries in Solr].
+
+[[OtherParsers-OldLuceneQueryParser]]
+== Old Lucene Query Parser
+
+`OldLuceneQParser` extends the `QParserPlugin` by parsing Solr's variant of Lucene's QueryParser syntax, including the deprecated sort specification after the query.
+
+Example:
+
+[source,text]
+----
+{!lucenePlusSort} myfield:foo +bar -baz;price asc
+----
+
+[[OtherParsers-PayloadQueryParsers]]
+== Payload Query Parsers
+
+These query parsers utilize payloads encoded on terms during indexing.
+
+The main query, for both of these parsers, is parsed straightforwardly from the field type's query analysis into a `SpanQuery`. The generated `SpanQuery` will be either a `SpanTermQuery` or an ordered, zero slop `SpanNearQuery`, depending on how many tokens are emitted. Payloads can be encoded on terms using either the `DelimitedPayloadTokenFilter` or the `NumericPayloadTokenFilter`. The payload using parsers are:
+
+* `PayloadScoreQParser`
+* `PayloadCheckQParser`
+
+[[OtherParsers-PayloadScoreParser]]
+=== Payload Score Parser
+
+`PayloadScoreQParser` incorporates each matching term's numeric (integer or float) payloads into the scores.
+
+This parser accepts the following parameters:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|`f` |Field to use (required)
+|`func` |Payload function: min, max, average (required)
+|`includeSpanScore` |If true, multiples computed payload factor by the score of the original query. If false, the computed payload factor is the score. (default: false)
+|===
+
+Example:
+
+[source,text]
+----
+{!payload_score f=my_field_dpf v=some_term func=max}
+----
+
+[[OtherParsers-PayloadCheckParser]]
+=== Payload Check Parser
+
+`PayloadCheckQParser` only matches when the matching terms also have the specified payloads.
+
+This parser accepts the following parameters:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|`f` |Field to use (required)
+|payloads a|
+A space-separated list of payloads that must match the query terms (required)
+
+Each specified payload will be encoded using the encoder determined from the field type and encoded accordingly for matching.
+
+`DelimitedPayloadTokenFilter` 'identity' encoded payloads also work here, as well as float and integer encoded ones.
+
+|===
+
+[source,text]
+----
+{!payload_check f=words_dps payloads="VERB NOUN"}searching stuff
+----
+
+[[OtherParsers-PrefixQueryParser]]
+== Prefix Query Parser
+
+`PrefixQParser` extends the `QParserPlugin` by creating a prefix query from the input value. Currently no analysis or value transformation is done to create this prefix query.
+
+The parameter is `f`, the field. The string after the prefix declaration is treated as a wildcard query.
+
+Example:
+
+[source,text]
+----
+{!prefix f=myfield}foo
+----
+
+This would be generally equivalent to the Lucene query parser expression `myfield:foo*`.
+
+[[OtherParsers-RawQueryParser]]
+== Raw Query Parser
+
+`RawQParser` extends the `QParserPlugin` by creating a term query from the input value without any text analysis or transformation. This is useful in debugging, or when raw terms are returned from the terms component (this is not the default).
+
+The only parameter is `f`, which defines the field to search.
+
+Example:
+
+[source,text]
+----
+{!raw f=myfield}Foo Bar
+----
+
+This example constructs the query: `TermQuery(Term("myfield","Foo Bar"))`.
+
+For easy filter construction to drill down in faceting, the <<OtherParsers-TermQueryParser,TermQParserPlugin>> is recommended.
+
+For full analysis on all fields, including text fields, you may want to use the <<OtherParsers-FieldQueryParser,FieldQParserPlugin>>.
+
+[[OtherParsers-Re-RankingQueryParser]]
+== Re-Ranking Query Parser
+
+The `ReRankQParserPlugin` is a special purpose parser for Re-Ranking the top results of a simple query using a more complex ranking query.
+
+Details about using the `ReRankQParserPlugin` can be found in the <<query-re-ranking.adoc#query-re-ranking,Query Re-Ranking>> section.
+
+[[OtherParsers-SimpleQueryParser]]
+== Simple Query Parser
+
+The Simple query parser in Solr is based on Lucene's SimpleQueryParser. This query parser is designed to allow users to enter queries however they want, and it will do its best to interpret the query and return results.
+
+This parser takes the following parameters:
+
+q.operators::
+Comma-separated list of names of parsing operators to enable. By default, all operations are enabled, and this parameter can be used to effectively disable specific operators as needed, by excluding them from the list. Passing an empty string with this parameter disables all operators.
++
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="15,20,50,15",options="header"]
+|===
+|Name |Operator |Description |Example query
+|`AND` |`+` |Specifies AND |`token1+token2`
+|`OR` |`\|` |Specifies OR |`token1\|token2`
+|`NOT` |`-` |Specifies NOT |`-token3`
+|`PREFIX` |`*` |Specifies a prefix query |`term*`
+|`PHRASE` |`"` |Creates a phrase |`"term1 term2"`
+|`PRECEDENCE` |`( )` |Specifies precedence; tokens inside the parenthesis will be analyzed first. Otherwise, normal order is left to right. |`token1 + (token2 \| token3)`
+|`ESCAPE` |`\` |Put it in front of operators to match them literally |`C\+\+`
+|`WHITESPACE` |space or `[\r\t\n]` a|Delimits tokens on whitespace. If not enabled, whitespace splitting will not be performed prior to analysis – usually most desirable.
+
+Not splitting whitespace is a unique feature of this parser that enables multi-word synonyms to work. However, it probably actually won't unless synonyms are configured to normalize instead of expand to all that match a given synonym. Such a configuration requires normalizing synonyms at both index time and query time. Solr's analysis screen can help here. |`term1 term2`
+|`FUZZY` a|
+`~`
+
+`~_N_`
+
+ a|
+At the end of terms, specifies a fuzzy query.
+
+"N" is optional and may be either "1" or "2" (the default)
+|`term~1`
+|`NEAR` |`~_N_` |At the end of phrases, specifies a NEAR query |`"term1 term2"~5`
+|===
+
+q.op::
+Defines the default operator to use if none is defined by the user. Allowed values are `AND` and `OR`. `OR` is used if none is specified.
+
+qf:: A list of query fields and boosts to use when building the query.
+
+df::
+Defines the default field if none is defined in the Schema, or overrides the default field if it is already defined.
+
+Any errors in syntax are ignored and the query parser will interpret queries as best it can. However, this can lead to odd results in some cases.
+
+[[OtherParsers-SpatialQueryParsers]]
+== Spatial Query Parsers
+
+There are two spatial QParsers in Solr: `geofilt` and `bbox`. But there are other ways to query spatially: using the `frange` parser with a distance function, using the standard (lucene) query parser with the range syntax to pick the corners of a rectangle, or with RPT and BBoxField you can use the standard query parser but use a special syntax within quotes that allows you to pick the spatial predicate.
+
+All these options are documented further in the section <<spatial-search.adoc#spatial-search,Spatial Search>>.
+
+[[OtherParsers-SurroundQueryParser]]
+== Surround Query Parser
+
+The `SurroundQParser` enables the Surround query syntax, which provides proximity search functionality. There are two positional operators: `w` creates an ordered span query and `n` creates an unordered one. Both operators take a numeric value to indicate distance between two terms. The default is 1, and the maximum is 99.
+
+Note that the query string is not analyzed in any way.
+
+Example:
+
+[source,text]
+----
+{!surround} 3w(foo, bar)
+----
+
+This example finds documents where the terms "foo" and "bar" are no more than 3 terms away from each other (i.e., no more than 2 terms between them).
+
+This query parser will also accept boolean operators (`AND`, `OR`, and `NOT`, in either upper- or lowercase), wildcards, quoting for phrase searches, and boosting. The `w` and `n` operators can also be expressed in upper- or lowercase.
+
+The non-unary operators (everything but `NOT`) support both infix `(a AND b AND c)` and prefix `AND(a, b, c)` notation.
+
+[[OtherParsers-SwitchQueryParser]]
+== Switch Query Parser
+
+`SwitchQParser` is a `QParserPlugin` that acts like a "switch" or "case" statement.
+
+The primary input string is trimmed and then prefixed with `case.` for use as a key to lookup a "switch case" in the parser's local params. If a matching local param is found the resulting param value will then be parsed as a subquery, and returned as the parse result.
+
+The `case` local param can be optionally be specified as a switch case to match missing (or blank) input strings. The `default` local param can optionally be specified as a default case to use if the input string does not match any other switch case local params. If default is not specified, then any input which does not match a switch case local param will result in a syntax error.
+
+In the examples below, the result of each query is "XXX":
+
+[source,text]
+----
+{!switch case.foo=XXX case.bar=zzz case.yak=qqq}foo
+----
+
+.The extra whitespace between `}` and `bar` is trimmed automatically.
+[source,text]
+----
+{!switch case.foo=qqq case.bar=XXX case.yak=zzz} bar
+----
+
+.The result will fallback to the default.
+[source,text]
+----
+{!switch case.foo=qqq case.bar=zzz default=XXX}asdf
+----
+
+.No input uses the value for `case` instead.
+[source,text]
+----
+{!switch case=XXX case.bar=zzz case.yak=qqq}
+----
+
+A practical usage of this `QParserPlugin`, is in specifying `appends` fq params in the configuration of a SearchHandler, to provide a fixed set of filter options for clients using custom parameter names.
+
+Using the example configuration below, clients can optionally specify the custom parameters `in_stock` and `shipping` to override the default filtering behavior, but are limited to the specific set of legal values (shipping=any|free, in_stock=yes|no|all).
+
+[source,xml]
+----
+<requestHandler name="/select" class="solr.SearchHandler">
+  <lst name="defaults">
+    <str name="in_stock">yes</str>
+    <str name="shipping">any</str>
+  </lst>
+  <lst name="appends">
+    <str name="fq">{!switch case.all='*:*'
+                            case.yes='inStock:true'
+                            case.no='inStock:false'
+                            v=$in_stock}</str>
+    <str name="fq">{!switch case.any='*:*'
+                            case.free='shipping_cost:0.0'
+                            v=$shipping}</str>
+  </lst>
+</requestHandler>
+----
+
+[[OtherParsers-TermQueryParser]]
+== Term Query Parser
+
+`TermQParser` extends the `QParserPlugin` by creating a single term query from the input value equivalent to `readableToIndexed()`. This is useful for generating filter queries from the external human readable terms returned by the faceting or terms components. The only parameter is `f`, for the field.
+
+Example:
+
+[source,text]
+----
+{!term f=weight}1.5
+----
+
+For text fields, no analysis is done since raw terms are already returned from the faceting and terms components. To apply analysis to text fields as well, see the <<OtherParsers-FieldQueryParser,Field Query Parser>>, above.
+
+If no analysis or transformation is desired for any type of field, see the <<OtherParsers-RawQueryParser,Raw Query Parser>>, above.
+
+[[OtherParsers-TermsQueryParser]]
+== Terms Query Parser
+
+`TermsQParser`, functions similarly to the <<OtherParsers-TermQueryParser,Term Query Parser>> but takes in multiple values separated by commas and returns documents matching any of the specified values.
+
+This can be useful for generating filter queries from the external human readable terms returned by the faceting or terms components, and may be more efficient in some cases than using the <<the-standard-query-parser.adoc#the-standard-query-parser,Standard Query Parser>> to generate an boolean query since the default implementation "```method```" avoids scoring.
+
+This query parser takes the following parameters:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Description
+|f |The field on which to search. Required.
+|separator |Separator to use when parsing the input. If set to " " (a single blank space), will trim additional white space from the input terms. Defaults to "`,`".
+|method |The internal query-building implementation: `termsFilter`, `booleanQuery`, `automaton`, or `docValuesTermsFilter`. Defaults to "```termsFilter```".
+|===
+
+Examples:
+
+[source,text]
+----
+{!terms f=tags}software,apache,solr,lucene
+----
+
+[source,text]
+----
+{!terms f=categoryId method=booleanQuery separator=" "}8 6 7 5309
+----
+
+[[OtherParsers-XMLQueryParser]]
+== XML Query Parser
+
+The {solr-javadocs}/solr-core/org/apache/solr/search/XmlQParserPlugin.html[XmlQParserPlugin] extends the {solr-javadocs}/solr-core/org/apache/solr/search/QParserPlugin.html[QParserPlugin] and supports the creation of queries from XML. Example:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[cols="30,70",options="header"]
+|===
+|Parameter |Value
+|defType |`xmlparser`
+|q a|
+[source,xml]
+<BooleanQuery fieldName="description"> <Clause occurs="must"> <TermQuery>shirt</TermQuery> </Clause> <Clause occurs="mustnot"> <TermQuery>plain</TermQuery> </Clause> <Clause occurs="should"> <TermQuery>cotton</TermQuery> </Clause> <Clause occurs="must"> <BooleanQuery fieldName="size"> <Clause occurs="should"> <TermsQuery>S M L</TermsQuery> </Clause> </BooleanQuery> </Clause> </BooleanQuery>
+|===
+
+The XmlQParser implementation uses the {solr-javadocs}/solr-core/org/apache/solr/search/SolrCoreParser.html[SolrCoreParser] class which extends Lucene's {lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/CoreParser.html[CoreParser] class. XML elements are mapped to {lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/QueryBuilder.html[QueryBuilder] classes as follows:
+
+// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+
+[width="50%",cols="30,70",options="header"]
+|===
+|XML element |QueryBuilder class
+|<BooleanQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/BooleanQueryBuilder.html[BooleanQueryBuilder]
+|<BoostingTermQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/BoostingTermBuilder.html[BoostingTermBuilder]
+|<ConstantScoreQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/ConstantScoreQueryBuilder.html[ConstantScoreQueryBuilder]
+|<DisjunctionMaxQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/DisjunctionMaxQueryBuilder.html[DisjunctionMaxQueryBuilder]
+|<MatchAllDocsQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/MatchAllDocsQueryBuilder.html[MatchAllDocsQueryBuilder]
+|<RangeQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/RangeQueryBuilder.html[RangeQueryBuilder]
+|<SpanFirst> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanFirstBuilder.html[SpanFirstBuilder]
+|<SpanNear> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanNearBuilder.html[SpanNearBuilder]
+|<SpanNot> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanNotBuilder.html[SpanNotBuilder]
+|<SpanOr> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanOrBuilder.html[SpanOrBuilder]
+|<SpanOrTerms> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanOrTermsBuilder.html[SpanOrTermsBuilder]
+|<SpanTerm> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/SpanTermBuilder.html[SpanTermBuilder]
+|<TermQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/TermQueryBuilder.html[TermQueryBuilder]
+|<TermsQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/TermsQueryBuilder.html[TermsQueryBuilder]
+|<UserQuery> |{lucene-javadocs}/queryparser/org/apache/lucene/queryparser/xml/builders/UserInputQueryBuilder.html[UserInputQueryBuilder]
+|<LegacyNumericRangeQuery> |LegacyNumericRangeQuery(Builder) is deprecated
+|===
+
+[[OtherParsers-CustomizingXMLQueryParser]]
+=== Customizing XML Query Parser
+
+You can configure your own custom query builders for additional XML elements. The custom builders need to extend the {solr-javadocs}/solr-core/org/apache/solr/search/SolrQueryBuilder.html[SolrQueryBuilder] or the {solr-javadocs}/solr-core/org/apache/solr/search/SolrSpanQueryBuilder.html[SolrSpanQueryBuilder] class. Example solrconfig.xml snippet:
+
+[source,xml]
+----
+<queryParser name="xmlparser" class="XmlQParserPlugin">
+  <str name="MyCustomQuery">com.mycompany.solr.search.MyCustomQueryBuilder</str>
+</queryParser>
+----

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/other-schema-elements.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/other-schema-elements.adoc b/solr/solr-ref-guide/src/other-schema-elements.adoc
new file mode 100644
index 0000000..42254eb
--- /dev/null
+++ b/solr/solr-ref-guide/src/other-schema-elements.adoc
@@ -0,0 +1,90 @@
+= Other Schema Elements
+:page-shortname: other-schema-elements
+:page-permalink: other-schema-elements.html
+
+This section describes several other important elements of `schema.xml` not covered in earlier sections.
+
+[[OtherSchemaElements-UniqueKey]]
+== Unique Key
+
+The `uniqueKey` element specifies which field is a unique identifier for documents. Although `uniqueKey` is not required, it is nearly always warranted by your application design. For example, `uniqueKey` should be used if you will ever update a document in the index.
+
+You can define the unique key field by naming it:
+
+[source,xml]
+----
+<uniqueKey>id</uniqueKey>
+----
+
+Schema defaults and `copyFields` cannot be used to populate the `uniqueKey` field. The `fieldType` of `uniqueKey` must not be analyzed. You can use `UUIDUpdateProcessorFactory` to have `uniqueKey` values generated automatically.
+
+Further, the operation will fail if the `uniqueKey` field is used, but is multivalued (or inherits the multivalue-ness from the `fieldtype`). However, `uniqueKey` will continue to work, as long as the field is properly used.
+
+
+[[OtherSchemaElements-DefaultSearchField_QueryOperator]]
+== Default Search Field & Query Operator
+
+Although they have been deprecated for quite some time, Solr still has support for Schema based configuration of a `<defaultSearchField/>` (which is superseded by the <<the-standard-query-parser.adoc#the-standard-query-parser,`df parameter`>>) and `<solrQueryParser defaultOperator="OR"/>` (which is superseded by the <<the-standard-query-parser.adoc#the-standard-query-parser,`q.op` parameter>>.
+
+If you have these options specified in your Schema, you are strongly encouraged to replace them with request parameters (or <<request-parameters-api.adoc#request-parameters-api,request parameter defaults>>) as support for them may be removed from future Solr release.
+
+[[OtherSchemaElements-Similarity]]
+== Similarity
+
+Similarity is a Lucene class used to score a document in searching.
+
+Each collection has one "global" Similarity, and by default Solr uses an implicit {solr-javadocs}/solr-core/org/apache/solr/search/similarities/SchemaSimilarityFactory.html[`SchemaSimilarityFactory`] which allows individual field types to be configured with a "per-type" specific Similarity and implicitly uses `BM25Similarity` for any field type which does not have an explicit Similarity.
+
+This default behavior can be overridden by declaring a top level `<similarity/>` element in your `schema.xml`, outside of any single field type. This similarity declaration can either refer directly to the name of a class with a no-argument constructor, such as in this example showing `BM25Similarity`:
+
+[source,xml]
+----
+<similarity class="solr.BM25SimilarityFactory"/>
+----
+
+or by referencing a `SimilarityFactory` implementation, which may take optional initialization parameters:
+
+[source,xml]
+----
+<similarity class="solr.DFRSimilarityFactory">
+  <str name="basicModel">P</str>
+  <str name="afterEffect">L</str>
+  <str name="normalization">H2</str>
+  <float name="c">7</float>
+</similarity>
+----
+
+In most cases, specifying global level similarity like this will cause an error if your `schema.xml` also includes field type specific `<similarity/>` declarations. One key exception to this is that you may explicitly declare a {solr-javadocs}/solr-core/org/apache/solr/search/similarities/SchemaSimilarityFactory.html[`SchemaSimilarityFactory`] and specify what that default behavior will be for all field types that do not declare an explicit Similarity using the name of field type (specified by `defaultSimFromFieldType`) that _is_ configured with a specific similarity:
+
+[source,xml]
+----
+<similarity class="solr.SchemaSimilarityFactory">
+  <str name="defaultSimFromFieldType">text_dfr</str>
+</similarity>
+<fieldType name="text_dfr" class="solr.TextField">
+  <analyzer ... />
+  <similarity class="solr.DFRSimilarityFactory">
+    <str name="basicModel">I(F)</str>
+    <str name="afterEffect">B</str>
+    <str name="normalization">H3</str>
+    <float name="mu">900</float>
+  </similarity>
+</fieldType>
+<fieldType name="text_ib" class="solr.TextField">
+  <analyzer ... />
+  <similarity class="solr.IBSimilarityFactory">
+    <str name="distribution">SPL</str>
+    <str name="lambda">DF</str>
+    <str name="normalization">H2</str>
+  </similarity>
+</fieldType>
+<fieldType name="text_other" class="solr.TextField">
+  <analyzer ... />
+</fieldType>
+----
+
+In the example above `IBSimilarityFactory` (using the Information-Based model) will be used for any fields of type `text_ib`, while `DFRSimilarityFactory` (divergence from random) will be used for any fields of type `text_dfr`, as well as any fields using a type that does not explicitly specify a `<similarity/>`.
+
+If `SchemaSimilarityFactory` is explicitly declared with out configuring a `defaultSimFromFieldType`, then `BM25Similarity` is implicitly used as the default.
+
+In addition to the various factories mentioned on this page, there are several other similarity implementations that can be used such as the `SweetSpotSimilarityFactory`, `ClassicSimilarityFactory`, etc.... For details, see the Solr Javadocs for the {solr-javadocs}/solr-core/org/apache/solr/schema/SimilarityFactory.html[similarity factories].

http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/95968c69/solr/solr-ref-guide/src/overview-of-documents-fields-and-schema-design.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/overview-of-documents-fields-and-schema-design.adoc b/solr/solr-ref-guide/src/overview-of-documents-fields-and-schema-design.adoc
new file mode 100644
index 0000000..6af1941
--- /dev/null
+++ b/solr/solr-ref-guide/src/overview-of-documents-fields-and-schema-design.adoc
@@ -0,0 +1,47 @@
+= Overview of Documents, Fields, and Schema Design
+:page-shortname: overview-of-documents-fields-and-schema-design
+:page-permalink: overview-of-documents-fields-and-schema-design.html
+
+The fundamental premise of Solr is simple. You give it a lot of information, then later you can ask it questions and find the piece of information you want. The part where you feed in all the information is called _indexing_ or _updating_. When you ask a question, it's called a _query_.
+
+One way to understand how Solr works is to think of a loose-leaf book of recipes. Every time you add a recipe to the book, you update the index at the back. You list each ingredient and the page number of the recipe you just added. Suppose you add one hundred recipes. Using the index, you can very quickly find all the recipes that use garbanzo beans, or artichokes, or coffee, as an ingredient. Using the index is much faster than looking through each recipe one by one. Imagine a book of one thousand recipes, or one million.
+
+Solr allows you to build an index with many different fields, or types of entries. The example above shows how to build an index with just one field, `ingredients`. You could have other fields in the index for the recipe's cooking style, like `Asian`, `Cajun`, or `vegan`, and you could have an index field for preparation times. Solr can answer questions like "What Cajun-style recipes that have blood oranges as an ingredient can be prepared in fewer than 30 minutes?"
+
+The schema is the place where you tell Solr how it should build indexes from input documents.
+
+== How Solr Sees the World
+
+Solr's basic unit of information is a _document_, which is a set of data that describes something. A recipe document would contain the ingredients, the instructions, the preparation time, the cooking time, the tools needed, and so on. A document about a person, for example, might contain the person's name, biography, favorite color, and shoe size. A document about a book could contain the title, author, year of publication, number of pages, and so on.
+
+In the Solr universe, documents are composed of _fields_, which are more specific pieces of information. Shoe size could be a field. First name and last name could be fields.
+
+Fields can contain different kinds of data. A name field, for example, is text (character data). A shoe size field might be a floating point number so that it could contain values like 6 and 9.5. Obviously, the definition of fields is flexible (you could define a shoe size field as a text field rather than a floating point number, for example), but if you define your fields correctly, Solr will be able to interpret them correctly and your users will get better results when they perform a query.
+
+You can tell Solr about the kind of data a field contains by specifying its _field type_. The field type tells Solr how to interpret the field and how it can be queried.
+
+When you add a document, Solr takes the information in the document's fields and adds that information to an index. When you perform a query, Solr can quickly consult the index and return the matching documents.
+
+== Field Analysis
+
+_Field analysis_ tells Solr what to do with incoming data when building an index. A more accurate name for this process would be _processing_ or even _digestion_, but the official name is _analysis_.
+
+Consider, for example, a biography field in a person document. Every word of the biography must be indexed so that you can quickly find people whose lives have had anything to do with ketchup, or dragonflies, or cryptography.
+
+However, a biography will likely contains lots of words you don't care about and don't want clogging up your index—words like "the", "a", "to", and so forth. Furthermore, suppose the biography contains the word "Ketchup", capitalized at the beginning of a sentence. If a user makes a query for "ketchup", you want Solr to tell you about the person even though the biography contains the capitalized word.
+
+The solution to both these problems is field analysis. For the biography field, you can tell Solr how to break apart the biography into words. You can tell Solr that you want to make all the words lower case, and you can tell Solr to remove accents marks.
+
+Field analysis is an important part of a field type. <<understanding-analyzers-tokenizers-and-filters.adoc#understanding-analyzers-tokenizers-and-filters,Understanding Analyzers, Tokenizers, and Filters>> is a detailed description of field analysis.
+
+== Solr's Schema File
+
+Solr stores details about the field types and fields it is expected to understand in a schema file. The name and location of this file may vary depending on how you initially configured Solr or if you modified it later.
+
+* `managed-schema` is the name for the schema file Solr uses by default to support making Schema changes at runtime via the <<schema-api.adoc#schema-api,Schema API>>, or <<schemaless-mode.adoc#schemaless-mode,Schemaless Mode>> features. You may <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,explicitly configure the managed schema features>> to use an alternative filename if you choose, but the contents of the files are still updated automatically by Solr.
+* `schema.xml` is the traditional name for a schema file which can be edited manually by users who use the <<schema-factory-definition-in-solrconfig.adoc#schema-factory-definition-in-solrconfig,`ClassicIndexSchemaFactory`>>.
+* If you are using SolrCloud you may not be able to find any file by these names on the local filesystem. You will only be able to see the schema through the Schema API (if enabled) or through the Solr Admin UI's <<cloud-screens.adoc#cloud-screens,Cloud Screens>>.
+
+Whichever name of the file in use in your installation, the structure of the file is not changed. However, the way you interact with the file will change. If you are using the managed schema, it is expected that you only interact with the file with the Schema API, and never make manual edits. If you do not use the managed schema, you will only be able to make manual edits to the file, the Schema API will not support any modifications.
+
+Note that if you are not using the Schema API yet you do use SolrCloud, you will need to interact with `schema.xml` through ZooKeeper using upconfig and downconfig commands to make a local copy and upload your changes. The options for doing this are described in <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script Reference>> and <<using-zookeeper-to-manage-configuration-files.adoc#using-zookeeper-to-manage-configuration-files,Using ZooKeeper to Manage Configuration Files>>.