You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ct...@apache.org on 2017/03/16 17:29:08 UTC
[21/26] lucene-solr:jira/solr-10290: SOLR-10290: Add .adoc files
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/distributed-requests.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/distributed-requests.adoc b/solr/solr-ref-guide/src/distributed-requests.adoc
new file mode 100644
index 0000000..1fa0326
--- /dev/null
+++ b/solr/solr-ref-guide/src/distributed-requests.adoc
@@ -0,0 +1,128 @@
+= Distributed Requests
+:page-shortname: distributed-requests
+:page-permalink: distributed-requests.html
+
+When a Solr node receives a search request, that request is routed behind the scenes to a replica of a shard that is part of the collection being searched. The chosen replica acts as an aggregator: it creates internal requests to randomly chosen replicas of every shard in the collection, coordinates the responses, issues any subsequent internal requests as needed (for example, to refine facets values, or request additional stored fields), and constructs the final response for the client.
+
+[[DistributedRequests-LimitingWhichShardsareQueried]]
+== Limiting Which Shards are Queried
+
+While one of the advantages of using SolrCloud is the ability to query very large collections distributed among various shards, in some cases <<shards-and-indexing-data-in-solrcloud.adoc#ShardsandIndexingDatainSolrCloud-DocumentRouting,you may know that you are only interested in results from a subset of your shards>>. You have the option of searching over all of your data or just parts of it.
+
+Querying all shards for a collection should look familiar; it's as though SolrCloud didn't even come into play:
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*
+----
+
+If, on the other hand, you wanted to search just one shard, you can specify that shard by its logical ID, as in:
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1
+----
+
+If you want to search a group of shard Ids, you can specify them together:
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,shard2
+----
+
+In both of the above examples, the shard Id(s) will be used to pick a random replica of that shard.
+
+Alternatively, you can specify the explict replicas you wish to use in place of a shard Ids:
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted,localhost:8983/solr/gettingstarted
+----
+
+Or you can specify a list of replicas to choose from for a single shard (for load balancing purposes) by using the pipe symbol (|):
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
+----
+
+And of course, you can specify a list of shards (seperated by commas) each defined by a list of replicas (seperated by pipes). In this example, 2 shards are queried, the first being a random replica from shard1, the second being a random replica from the explicit pipe delimited list:
+
+[source,java]
+----
+http://localhost:8983/solr/gettingstarted/select?q=*:*&shards=shard1,localhost:7574/solr/gettingstarted|localhost:7500/solr/gettingstarted
+----
+
+[[DistributedRequests-ConfiguringtheShardHandlerFactory]]
+== Configuring the ShardHandlerFactory
+
+You can directly configure aspects of the concurrency and thread-pooling used within distributed search in Solr. This allows for finer grained control and you can tune it to target your own specific requirements. The default configuration favors throughput over latency.
+
+To configure the standard handler, provide a configuration like this in the solrconfig.xml:
+
+[source,xml]
+----
+<requestHandler name="standard" class="solr.SearchHandler" default="true">
+ <!-- other params go here -->
+ <shardHandler class="HttpShardHandlerFactory">
+ <int name="socketTimeOut">1000</int>
+ <int name="connTimeOut">5000</int>
+ </shardHandler>
+</requestHandler>
+----
+
+The parameters that can be specified are as follows:
+
+// TODO: This table has cells that won't work with PDF: https://github.com/ctargett/refguide-asciidoc-poc/issues/13
+
+[width="100%",cols="34%,33%,33%",options="header",]
+|===
+|Parameter |Default |Explanation
+|`socketTimeout` |0 (use OS default) |The amount of time in ms that a socket is allowed to wait.
+|`connTimeout` |0 (use OS default) |The amount of time in ms that is accepted for binding / connecting a socket
+|`maxConnectionsPerHost` |20 |The maximum number of concurrent connections that is made to each individual shard in a distributed search.
+|`maxConnections` a|
+....
+10000
+....
+
+ |The total maximum number of concurrent connections in distributed searches.
+|`corePoolSize` |0 |The retained lowest limit on the number of threads used in coordinating distributed search.
+|`maximumPoolSize` |Integer.MAX_VALUE |The maximum number of threads used for coordinating distributed search.
+|`maxThreadIdleTime` |5 seconds |The amount of time to wait for before threads are scaled back in response to a reduction in load.
+|`sizeOfQueue` |-1 |If specified, the thread pool will use a backing queue instead of a direct handoff buffer. High throughput systems will want to configure this to be a direct hand off (with -1). Systems that desire better latency will want to configure a reasonable size of queue to handle variations in requests.
+|`fairnessPolicy` |false |Chooses the JVM specifics dealing with fair policy queuing, if enabled distributed searches will be handled in a First in First out fashion at a cost to throughput. If disabled throughput will be favored over latency.
+|===
+
+// OLD_CONFLUENCE_ID: DistributedRequests-ConfiguringstatsCache(DistributedIDF)
+
+[[DistributedRequests-ConfiguringstatsCache_DistributedIDF_]]
+== Configuring statsCache (Distributed IDF)
+
+Document and term statistics are needed in order to calculate relevancy. Solr provides four implementations out of the box when it comes to document stats calculation:
+
+* `LocalStatsCache`: This only uses local term and document statistics to compute relevance. In cases with uniform term distribution across shards, this works reasonably well.This option is the default if no `<statsCache>` is configured.
+* `ExactStatsCache`: This implementation uses global values (across the collection) for document frequency.
+* `ExactSharedStatsCache`: This is exactly like the exact stats cache in its functionality but the global stats are reused for subsequent requests with the same terms.
+* `LRUStatsCache`: This implementation uses an LRU cache to hold global stats, which are shared between requests.
+
+The implementation can be selected by setting `<statsCache>` in `solrconfig.xml`. For example, the following line makes Solr use the `ExactStatsCache` implementation:
+
+[source,xml]
+----
+<statsCache class="org.apache.solr.search.stats.ExactStatsCache"/>
+----
+
+[[DistributedRequests-AvoidingDistributedDeadlock]]
+== Avoiding Distributed Deadlock
+
+Each shard serves top-level query requests and then makes sub-requests to all of the other shards. Care should be taken to ensure that the max number of threads serving HTTP requests is greater than the possible number of requests from both top-level clients and other shards. If this is not the case, the configuration may result in a distributed deadlock.
+
+For example, a deadlock might occur in the case of two shards, each with just a single thread to service HTTP requests. Both threads could receive a top-level request concurrently, and make sub-requests to each other. Because there are no more remaining threads to service requests, the incoming requests will be blocked until the other pending requests are finished, but they will not finish since they are waiting for the sub-requests. By ensuring that Solr is configured to handle a sufficient number of threads, you can avoid deadlock situations like this.
+
+[[DistributedRequests-PreferLocalShards]]
+== Prefer Local Shards
+
+Solr allows you to pass an optional boolean parameter named `preferLocalShards` to indicate that a distributed query should prefer local replicas of a shard when available. In other words, if a query includes `preferLocalShards=true`, then the query controller will look for local replicas to service the query instead of selecting replicas at random from across the cluster. This is useful when a query requests many fields or large fields to be returned per document because it avoids moving large amounts of data over the network when it is available locally. In addition, this feature can be useful for minimizing the impact of a problematic replica with degraded performance, as it reduces the likelihood that the degraded replica will be hit by other healthy replicas.
+
+Lastly, it follows that the value of this feature diminishes as the number of shards in a collection increases because the query controller will have to direct the query to non-local replicas for most of the shards. In other words, this feature is mostly useful for optimizing queries directed towards collections with a small number of shards and many replicas. Also, this option should only be used if you are load balancing requests across all nodes that host replicas for the collection you are querying, as Solr's CloudSolrClient will do. If not load-balancing, this feature can introduce a hotspot in the cluster since queries won't be evenly distributed across the cluster.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/distributed-search-with-index-sharding.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/distributed-search-with-index-sharding.adoc b/solr/solr-ref-guide/src/distributed-search-with-index-sharding.adoc
new file mode 100644
index 0000000..e8e27ec
--- /dev/null
+++ b/solr/solr-ref-guide/src/distributed-search-with-index-sharding.adoc
@@ -0,0 +1,165 @@
+= Distributed Search with Index Sharding
+:page-shortname: distributed-search-with-index-sharding
+:page-permalink: distributed-search-with-index-sharding.html
+
+It is highly recommended that you use <<solrcloud.adoc#solrcloud,SolrCloud>> when needing to scale up or scale out. The setup described below is legacy and was used prior to the existence of SolrCloud. SolrCloud provides for a truly distributed set of features with support for things like automatic routing, leader election, optimistic concurrency and other sanity checks that are expected out of a distributed system.
+
+Everything on this page is specific to legacy setup of distributed search. Users trying out SolrCloud should not follow any of the steps or information below.
+
+Update reorders (i.e., replica A may see update X then Y, and replica B may see update Y then X). *deleteByQuery* also handles reorders the same way, to ensure replicas are consistent. All replicas of a shard are consistent, even if the updates arrive in a different order on different replicas.
+
+[[DistributedSearchwithIndexSharding-DistributingDocumentsacrossShards]]
+== Distributing Documents across Shards
+
+When not using SolrCloud, it is up to you to get all your documents indexed on each shard of your server farm. Solr supports distributed indexing (routing) in its true form only in the SolrCloud mode.
+
+In the legacy distributed mode, Solr does not calculate universal term/doc frequencies. For most large-scale implementations, it is not likely to matter that Solr calculates TF/IDF at the shard level. However, if your collection is heavily skewed in its distribution across servers, you may find misleading relevancy results in your searches. In general, it is probably best to randomly distribute documents to your shards.
+
+[[DistributedSearchwithIndexSharding-ExecutingDistributedSearcheswiththeshardsParameter]]
+== Executing Distributed Searches with the `shards` Parameter
+
+If a query request includes the `shards` parameter, the Solr server distributes the request across all the shards listed as arguments to the parameter. The `shards` parameter uses this syntax:
+
+`__host__` `:` `__port__` `/` `__base_url__` `[,` `__host__` `:` `__port__` `/` `__base_url__` `]*`
+
+For example, the `shards` parameter below causes the search to be distributed across two Solr servers: *solr1* and **solr2**, both of which are running on port 8983:
+
+` http://localhost:8983/solr/core1/select?shards=solr1:8983/solr/core1,solr2:8983/solr/core1&indent=true&q=ipod+solr `
+
+Rather than require users to include the shards parameter explicitly, it is usually preferred to configure this parameter as a default in the RequestHandler section of `solrconfig.xml`.
+
+[IMPORTANT]
+====
+
+Do not add the `shards` parameter to the standard requestHandler; otherwise, search queries may enter an infinite loop. Instead, define a new requestHandler that uses the `shards` parameter, and pass distributed search requests to that handler.
+
+====
+
+With Legacy mode, only query requests are distributed. This includes requests to the SearchHandler (or any handler extending from `org.apache.solr.handler.component.SearchHandler`) using standard components that support distributed search.
+
+As in SolrCloud mode, when `shards.info=true`, distributed responses will include information about the shard (where each shard represents a logically different index or physical location)
+
+The following components support distributed search:
+
+* The *Query* component, which returns documents matching a query
+* The *Facet* component, which processes facet.query and facet.field requests where facets are sorted by count (the default).
+* The *Highlighting* component, which enables Solr to include "highlighted" matches in field values.
+* The *Stats* component, which returns simple statistics for numeric fields within the DocSet.
+* The *Debug* component, which helps with debugging.
+
+[[DistributedSearchwithIndexSharding-LimitationstoDistributedSearch]]
+== Limitations to Distributed Search
+
+Distributed searching in Solr has the following limitations:
+
+* Each document indexed must have a unique key.
+* If Solr discovers duplicate document IDs, Solr selects the first document and discards subsequent ones.
+* The index for distributed searching may become momentarily out of sync if a commit happens between the first and second phase of the distributed search. This might cause a situation where a document that once matched a query and was subsequently changed may no longer match the query but will still be retrieved. This situation is expected to be quite rare, however, and is only possible for a single query request.
+* The number of shards is limited by number of characters allowed for GET method's URI; most Web servers generally support at least 4000 characters, but many servers limit URI length to reduce their vulnerability to Denial of Service (DoS) attacks.
+* Shard information can be returned with each document in a distributed search by including `fl=id, [shard]` in the search request. This returns the shard URL.
+* In a distributed search, the data directory from the core descriptor overrides any data directory in `solrconfig.xml.`
+* Update commands may be sent to any server with distributed indexing configured correctly. Document adds and deletes are forwarded to the appropriate server/shard based on a hash of the unique document id. *commit* commands and *deleteByQuery* commands are sent to every server in `shards`.
+
+Formerly a limitation was that TF/IDF relevancy computations only used shard-local statistics. This is still the case by default. If your data isn't randomly distributed, or if you want more exact statistics, then remember to configure the ExactStatsCache.
+
+[[DistributedSearchwithIndexSharding-AvoidingDistributedDeadlock]]
+== Avoiding Distributed Deadlock
+
+Like in SolrCloud mode, inter-shard requests could lead to a distributed deadlock. It can be avoided by following the instructions <<distributed-requests.adoc#distributed-requests,here>>.
+
+[[DistributedSearchwithIndexSharding-TestingIndexShardingonTwoLocalServers]]
+== Testing Index Sharding on Two Local Servers
+
+For simple functionality testing, it's easiest to just set up two local Solr servers on different ports. (In a production environment, of course, these servers would be deployed on separate machines.)
+
+1. Make two Solr home directories:
++
+[source,java]
+----
+mkdir example/nodes
+mkdir example/nodes/node1
+# Copy solr.xml into this solr.home
+cp server/solr/solr.xml example/nodes/node1/.
+# Repeat the above steps for the second node
+mkdir example/nodes/node2
+cp server/solr/solr.xml example/nodes/node2/.
+----
+2. Start the two Solr instances
++
+[source,java]
+----
+# Start first node on port 8983
+bin/solr start -s example/nodes/node1 -p 8983
+
+# Start second node on port 8984
+bin/solr start -s example/nodes/node2 -p 8984
+----
+3. Create a core on both the nodes with the sample_techproducts_configs.
++
+[source,java]
+----
+bin/solr create_core -c core1 -p 8983 -d sample_techproducts_configs
+# Create a core on the Solr node running on port 8984
+bin/solr create_core -c core1 -p 8984 -d sample_techproducts_configs
+----
+4. In the third window, index an example document to each of the server:
++
+[source,java]
+----
+bin/post -c core1 example/exampledocs/monitor.xml -port 8983
+
+bin/post -c core1 example/exampledocs/monitor2.xml -port 8984
+----
+5. Search on the node on port 8983:
++
+[source,bash]
+----
+curl http://localhost:8983/solr/core1/select?q=*:*&wt=xml&indent=true
+----
++
+This should bring back one document.
++
+Search on the node on port 8984:
++
+[source,bash]
+----
+curl http://localhost:8984/solr/core1/select?q=*:*&wt=xml&indent=true
+----
++
+This should also bring back a single document.
++
+Now do a distributed search across both servers with your browser or `curl.` In the example below, an extra parameter 'fl' is passed to restrict the returned fields to id and name.
++
+[source,bash]
+----
+curl http://localhost:8983/solr/core1/select?q=*:*&indent=true&shards=localhost:8983/solr/core1,localhost:8984/solr/core1&fl=id,name
+----
++
+This should contain both the documents as shown below:
++
+[source,java]
+----
+<response>
+ <lst name="responseHeader">
+ <int name="status">0</int>
+ <int name="QTime">8</int>
+ <lst name="params">
+ <str name="q">*:*</str>
+ <str name="shards">localhost:8983/solr/core1,localhost:8984/solr/core1</str>
+ <str name="indent">true</str>
+ <str name="fl">id,name</str>
+ <str name="wt">xml</str>
+ </lst>
+ </lst>
+ <result name="response" numFound="2" start="0" maxScore="1.0">
+ <doc>
+ <str name="id">3007WFP</str>
+ <str name="name">Dell Widescreen UltraSharp 3007WFP</str>
+ </doc>
+ <doc>
+ <str name="id">VA902B</str>
+ <str name="name">ViewSonic VA902B - flat panel display - TFT - 19"</str>
+ </doc>
+ </result>
+</response>
+----
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/documents-fields-and-schema-design.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/documents-fields-and-schema-design.adoc b/solr/solr-ref-guide/src/documents-fields-and-schema-design.adoc
new file mode 100644
index 0000000..15b2164
--- /dev/null
+++ b/solr/solr-ref-guide/src/documents-fields-and-schema-design.adoc
@@ -0,0 +1,28 @@
+= Documents, Fields, and Schema Design
+:page-shortname: documents-fields-and-schema-design
+:page-permalink: documents-fields-and-schema-design.html
+:page-children: overview-of-documents-fields-and-schema-design, solr-field-types, defining-fields, copying-fields, dynamic-fields, other-schema-elements, schema-api, putting-the-pieces-together, docvalues, schemaless-mode
+
+This section discusses how Solr organizes its data into documents and fields, as well as how to work with a schema in Solr.
+
+This section includes the following topics:
+
+<<overview-of-documents-fields-and-schema-design.adoc#overview-of-documents-fields-and-schema-design,Overview of Documents, Fields, and Schema Design>>: An introduction to the concepts covered in this section.
+
+<<solr-field-types.adoc#solr-field-types,Solr Field Types>>: Detailed information about field types in Solr, including the field types in the default Solr schema.
+
+<<defining-fields.adoc#defining-fields,Defining Fields>>: Describes how to define fields in Solr.
+
+<<copying-fields.adoc#copying-fields,Copying Fields>>: Describes how to populate fields with data copied from another field.
+
+<<dynamic-fields.adoc#dynamic-fields,Dynamic Fields>>: Information about using dynamic fields in order to catch and index fields that do not exactly conform to other field definitions in your schema.
+
+<<schema-api.adoc#schema-api,Schema API>>: Use curl commands to read various parts of a schema or create new fields and copyField rules.
+
+<<other-schema-elements.adoc#other-schema-elements,Other Schema Elements>>: Describes other important elements in the Solr schema.
+
+<<putting-the-pieces-together.adoc#putting-the-pieces-together,Putting the Pieces Together>>: A higher-level view of the Solr schema and how its elements work together.
+
+<<docvalues.adoc#docvalues,DocValues>>: Describes how to create a docValues index for faster lookups.
+
+<<schemaless-mode.adoc#schemaless-mode,Schemaless Mode>>: Automatically add previously unknown schema fields using value-based field type guessing.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/documents-screen.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/documents-screen.adoc b/solr/solr-ref-guide/src/documents-screen.adoc
new file mode 100644
index 0000000..dda091a
--- /dev/null
+++ b/solr/solr-ref-guide/src/documents-screen.adoc
@@ -0,0 +1,70 @@
+= Documents Screen
+:page-shortname: documents-screen
+:page-permalink: documents-screen.html
+
+The Documents screen provides a simple form allowing you to execute various Solr indexing commands in a variety of formats directly from the browser.
+
+image::images/documents-screen/documents_add_screen.png[image,height=400]
+
+
+The screen allows you to:
+
+* Copy documents in JSON, CSV or XML and submit them to the index
+* Upload documents (in JSON, CSV or XML)
+* Construct documents by selecting fields and field values
+
+The first step is to define the RequestHandler to use (aka, 'qt'). By default `/update` will be defined. To use Solr Cell, for example, change the request handler to `/update/extract`.
+
+Then choose the Document Type to define the type of document to load. The remaining parameters will change depending on the document type selected.
+
+[[DocumentsScreen-JSON]]
+=== JSON
+
+When using the JSON document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper JSON format.
+
+Then you can choose when documents should be added to the index (Commit Within), whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped), and, finally, if a document boost should be applied.
+
+This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.
+
+[[DocumentsScreen-CSV]]
+=== CSV
+
+When using the CSV document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper CSV format, with columns delimited and one row per document.
+
+Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped).
+
+[[DocumentsScreen-DocumentBuilder]]
+=== Document Builder
+
+The Document Builder provides a wizard-like interface to enter fields of a document
+
+[[DocumentsScreen-FileUpload]]
+=== File Upload
+
+The File Upload option allows choosing a prepared file and uploading it. If using only `/update` for the Request-Handler option, you will be limited to XML, CSV, and JSON.
+
+However, to use the ExtractingRequestHandler (aka Solr Cell), you can modify the Request-Handler to `/update/extract`. You must have this defined in your `solrconfig.xml` file, with your desired defaults. You should also update the `&literal.id` shown in the Extracting Req. Handler Params so the file chosen is given a unique id.
+
+Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped).
+
+[[DocumentsScreen-SolrCommand]]
+=== Solr Command
+
+The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit and optimize commands on the index.
+
+The documents should be structured as they would be if using `/update` on the command line.
+
+[[DocumentsScreen-XML]]
+=== XML
+
+When using the XML document type, the functionality is similar to using a requestHandler on the command line. Instead of putting the documents in a curl command, they can instead be input into the Document entry box. The document structure should still be in proper Solr XML format, with each document separated by `<doc>` tags and each field defined.
+
+Then you can choose when documents should be added to the index (Commit Within), and whether existing documents should be overwritten with incoming documents with the same id (if this is not **true**, then the incoming documents will be dropped).
+
+This option will only add or overwrite documents to the index; for other update tasks, see the <<DocumentsScreen-SolrCommand,Solr Command>> option.
+
+[[DocumentsScreen-RelatedTopics]]
+== Related Topics
+
+* <<uploading-data-with-index-handlers.adoc#uploading-data-with-index-handlers,Uploading Data with Index Handlers>>
+* <<uploading-data-with-solr-cell-using-apache-tika.adoc#uploading-data-with-solr-cell-using-apache-tika,Uploading Data with Solr Cell using Apache Tika>>
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/docvalues.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/docvalues.adoc b/solr/solr-ref-guide/src/docvalues.adoc
new file mode 100644
index 0000000..8dd4d4b
--- /dev/null
+++ b/solr/solr-ref-guide/src/docvalues.adoc
@@ -0,0 +1,92 @@
+= DocValues
+:page-shortname: docvalues
+:page-permalink: docvalues.html
+
+DocValues are a way of recording field values internally that is more efficient for some purposes, such as sorting and faceting, than traditional indexing.
+
+// OLD_CONFLUENCE_ID: DocValues-WhyDocValues?
+
+[[DocValues-WhyDocValues_]]
+== Why DocValues?
+
+The standard way that Solr builds the index is with an __inverted index__. This style builds a list of terms found in all the documents in the index and next to each term is a list of documents that the term appears in (as well as how many times the term appears in that document). This makes search very fast - since users search by terms, having a ready list of term-to-document values makes the query process faster.
+
+For other features that we now commonly associate with search, such as sorting, faceting, and highlighting, this approach is not very efficient. The faceting engine, for example, must look up each term that appears in each document that will make up the result set and pull the document IDs in order to build the facet list. In Solr, this is maintained in memory, and can be slow to load (depending on the number of documents, terms, etc.).
+
+In Lucene 4.0, a new approach was introduced. DocValue fields are now column-oriented fields with a document-to-value mapping built at index time. This approach promises to relieve some of the memory requirements of the fieldCache and make lookups for faceting, sorting, and grouping much faster.
+
+[[DocValues-EnablingDocValues]]
+== Enabling DocValues
+
+To use docValues, you only need to enable it for a field that you will use it with. As with all schema design, you need to define a field type and then define fields of that type with docValues enabled. All of these actions are done in `schema.xml`.
+
+Enabling a field for docValues only requires adding `docValues="true"` to the field (or field type) definition, as in this example from the `schema.xml` of Solr's `sample_techproducts_configs` <<config-sets.adoc#config-sets,config set>>:
+
+[source,xml]
+----
+<field name="manu_exact" type="string" indexed="false" stored="false" docValues="true" />
+----
+
+[IMPORTANT]
+====
+
+If you have already indexed data into your Solr index, you will need to completely re-index your content after changing your field definitions in `schema.xml` in order to successfully use docValues.
+
+====
+
+DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are:
+
+* `StrField` and `UUIDField`.
+** If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type.
+** If the field is multi-valued, Lucene will use the SORTED_SET type.
+* Any `Trie*` numeric fields, date fields and `EnumField`.
+** If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type.
+** If the field is multi-valued, Lucene will use the SORTED_SET type.
+* Boolean fields
+
+These Lucene types are related to how the values are sorted and stored.
+
+There are two implications of multi-valued DocValues being stored as SORTED_SET types that should be kept in mind when combined with /export (and, by extension, <<streaming-expressions.adoc#streaming-expressions,Streaming Expression>>-based functionality):
+
+1. Values are returned in sorted order rather than the original input order.
+2. If multiple, identical entries are in the field in a _single_ document, only one will be returned for that document.
+
+There is an additional configuration option available, which is to modify the `docValuesFormat` <<field-type-definitions-and-properties.adoc#FieldTypeDefinitionsandProperties-docValuesFormat,used by the field type>>. The default implementation employs a mixture of loading some things into memory and keeping some on disk. In some cases, however, you may choose to specify an alternative {lucene-javadocs}/core/org/apache/lucene/codecs/DocValuesFormat.html[DocValuesFormat implementation]. For example, you could choose to keep everything in memory by specifying `docValuesFormat="Memory"` on a field type:
+
+[source,xml]
+----
+<fieldType name="string_in_mem_dv" class="solr.StrField" docValues="true" docValuesFormat="Memory" />
+----
+
+Please note that the `docValuesFormat` option may change in future releases.
+
+[NOTE]
+====
+
+Lucene index back-compatibility is only supported for the default codec. If you choose to customize the `docValuesFormat` in your schema.xml, upgrading to a future version of Solr may require you to either switch back to the default codec and optimize your index to rewrite it into the default codec before upgrading, or re-build your entire index from scratch after upgrading.
+
+====
+
+[[DocValues-UsingDocValues]]
+== Using DocValues
+
+// OLD_CONFLUENCE_ID: DocValues-Sorting,Faceting&Functions
+
+[[DocValues-Sorting_Faceting_Functions]]
+=== Sorting, Faceting & Functions
+
+If `docValues="true"` for a field, then DocValues will automatically be used any time the field is used for <<common-query-parameters.adoc#CommonQueryParameters-ThesortParameter,sorting>>, <<faceting.adoc#faceting,faceting>> or <<function-queries.adoc#function-queries,Function Queries>>.
+
+[[DocValues-RetrievingDocValuesDuringSearch]]
+=== Retrieving DocValues During Search
+
+Field values retrieved during search queries are typically returned from stored values. However, non-stored docValues fields will be also returned along with other stored fields when all fields (or pattern matching globs) are specified to be returned (e.g. "`fl=*`") for search queries depending on the effective value of the `useDocValuesAsStored` parameter for each field. For schema versions >= 1.6, the implicit default is `useDocValuesAsStored="true"`. See <<field-type-definitions-and-properties.adoc#field-type-definitions-and-properties,Field Type Definitions and Properties>> & <<defining-fields.adoc#defining-fields,Defining Fields>> for more details.
+
+When `useDocValuesAsStored="false"`, non-stored DocValues fields can still be explicitly requested by name in the <<common-query-parameters.adoc#CommonQueryParameters-Thefl_FieldList_Parameter,fl param>>, but will not match glob patterns (`"*"`). Note that returning DocValues along with "regular" stored fields at query time has performance implications that stored fields may not because DocValues are column-oriented and may therefore incur additional cost to retrieve for each returned document. Also note that while returning non-stored fields from DocValues, the values of a multi-valued field are returned in sorted order (and not insertion order). If you require the multi-valued fields to be returned in the original insertion order, then make your multi-valued field as stored (such a change requires re-indexing).
+
+In cases where the query is returning _only_ docValues fields performance may improve since returning stored fields requires disk reads and decompression whereas returning docValues fields in the fl list only requires memory access.
+
+When retrieving fields from their docValues form, two important differences between regular stored fields and docValues fields must be understood:
+
+1. Order is _not_ preserved. For simply retrieving stored fields, the insertion order is the return order. For docValues, it is the _sorted_ order.
+2. Multiple identical entries are collapsed into a single value. Thus if I insert values 4, 5, 2, 4, 1, my return will be 1, 2, 4, 5.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/dynamic-fields.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/dynamic-fields.adoc b/solr/solr-ref-guide/src/dynamic-fields.adoc
new file mode 100644
index 0000000..3c79fa1
--- /dev/null
+++ b/solr/solr-ref-guide/src/dynamic-fields.adoc
@@ -0,0 +1,23 @@
+= Dynamic Fields
+:page-shortname: dynamic-fields
+:page-permalink: dynamic-fields.html
+
+_Dynamic fields_ allow Solr to index fields that you did not explicitly define in your schema. This is useful if you discover you have forgotten to define one or more fields. Dynamic fields can make your application less brittle by providing some flexibility in the documents you can add to Solr.
+
+A dynamic field is just like a regular field except it has a name with a wildcard in it. When you are indexing documents, a field that does not match any explicitly defined fields can be matched with a dynamic field.
+
+For example, suppose your schema includes a dynamic field with a name of `*_i`. If you attempt to index a document with a `cost_i` field, but no explicit `cost_i` field is defined in the schema, then the `cost_i` field will have the field type and analysis defined for `*_i`.
+
+Like regular fields, dynamic fields have a name, a field type, and options.
+
+[source,xml]
+----
+<dynamicField name="*_i" type="int" indexed="true" stored="true"/>
+----
+
+It is recommended that you include basic dynamic field mappings (like that shown above) in your `schema.xml`. The mappings can be very useful.
+
+[[DynamicFields-RelatedTopics]]
+== Related Topics
+
+* http://wiki.apache.org/solr/SchemaXml#Dynamic_fields[SchemaXML-Dynamic Fields]
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/enabling-ssl.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/enabling-ssl.adoc b/solr/solr-ref-guide/src/enabling-ssl.adoc
new file mode 100644
index 0000000..16ce746
--- /dev/null
+++ b/solr/solr-ref-guide/src/enabling-ssl.adoc
@@ -0,0 +1,375 @@
+= Enabling SSL
+:page-shortname: enabling-ssl
+:page-permalink: enabling-ssl.html
+
+Both SolrCloud and single-node Solr can encrypt communications to and from clients, and in SolrCloud between nodes, with SSL. This section describes enabling SSL with the example Jetty server using a self-signed certificate.
+
+For background on SSL certificates and keys, see http://www.tldp.org/HOWTO/SSL-Certificates-HOWTO/.
+
+[[EnablingSSL-BasicSSLSetup]]
+== Basic SSL Setup
+
+[[EnablingSSL-Generateaself-signedcertificateandakey]]
+=== Generate a self-signed certificate and a key
+
+To generate a self-signed certificate and a single key that will be used to authenticate both the server and the client, we'll use the JDK https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html[`keytool`] command and create a separate keystore. This keystore will also be used as a truststore below. It's possible to use the keystore that comes with the JDK for these purposes, and to use a separate truststore, but those options aren't covered here.
+
+Run the commands below in the `server/etc/` directory in the binary Solr distribution. It's assumed that you have the JDK `keytool` utility on your `PATH`, and that `openssl` is also on your `PATH`. See https://www.openssl.org/related/binaries.html for OpenSSL binaries for Windows and Solaris.
+
+The "`-ext SAN=...`" `keytool` option allows you to specify all the DNS names and/or IP addresses that will be allowed during hostname verification (but see below for how to skip hostname verification between Solr nodes so that you don't have to specify all hosts here). In addition to `localhost` and `127.0.0.1`, this example includes a LAN IP address `192.168.1.3` for the machine the Solr nodes will be running on:
+
+[source,text]
+----
+keytool -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass secret -storepass secret -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:localhost,IP:192.168.1.3,IP:127.0.0.1 -dname "CN=localhost, OU=Organizational Unit, O=Organization, L=Location, ST=State, C=Country"
+----
+
+The above command will create a keystore file named `solr-ssl.keystore.jks` in the current directory.
+
+[[EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL]]
+=== Convert the certificate and key to PEM format for use with cURL
+
+cURL isn't capable of using JKS formatted keystores, so the JKS keystore needs to be converted to PEM format, which cURL understands.
+
+First convert the JKS keystore into PKCS12 format using `keytool`:
+
+[source,text]
+----
+keytool -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12
+----
+
+The keytool application will prompt you to create a destination keystore password and for the source keystore password, which was set when creating the keystore ("secret" in the example shown above).
+
+Next convert the PKCS12 format keystore, including both the certificate and the key, into PEM format using the http://www.openssl.org[`openssl`] command:
+
+[source,text]
+----
+openssl pkcs12 -in solr-ssl.keystore.p12 -out solr-ssl.pem
+----
+
+If you want to use cURL on OS X Yosemite (10.10), you'll need to create a certificate-only version of the PEM format, as follows:
+
+[source,text]
+----
+openssl pkcs12 -nokeys -in solr-ssl.keystore.p12 -out solr-ssl.cacert.pem
+----
+
+[[EnablingSSL-SetcommonSSLrelatedsystemproperties]]
+=== Set common SSL related system properties
+
+The Solr Control Script is already setup to pass SSL-related Java system properties to the JVM. To activate the SSL settings, uncomment and update the set of properties beginning with SOLR_SSL_* in `bin/solr.in.sh`. (or `bin\solr.in.cmd` on Windows). Note, if you setup Solr as a service on Linux using the steps outlined in <<taking-solr-to-production.adoc#taking-solr-to-production,Taking Solr to Production>>, then make these changes in `/var/solr/solr.in.sh` instead.
+
+*bin/solr.in.sh example SOLR_SSL_* configuration*
+
+[source,text]
+----
+SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
+SOLR_SSL_KEY_STORE_PASSWORD=secret
+SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
+SOLR_SSL_TRUST_STORE_PASSWORD=secret
+# Require clients to authenticate
+SOLR_SSL_NEED_CLIENT_AUTH=false
+# Enable clients to authenticate (but not require)
+SOLR_SSL_WANT_CLIENT_AUTH=false
+# Define Key Store type if necessary
+SOLR_SSL_KEY_STORE_TYPE=JKS
+SOLR_SSL_TRUST_STORE_TYPE=JKS
+----
+
+When you start Solr, the `bin/solr` script includes the settings in `bin/solr.in.sh` and will pass these SSL-related system properties to the JVM.
+
+.Client Authentication Settings
+[WARNING]
+====
+
+Enable either SOLR_SSL_NEED_CLIENT_AUTH or SOLR_SSL_WANT_CLIENT_AUTH but not both at the same time. They are mutually exclusive and Jetty will select one of them which may not be what you expect.
+
+====
+
+Similarly, when you start Solr on Windows, the `bin\solr.cmd` script includes the settings in `bin\solr.in.cmd` - uncomment and update the set of properties beginning with `SOLR_SSL_*` to pass these SSL-related system properties to the JVM:
+
+*bin\solr.in.cmd example SOLR_SSL_* configuration*
+
+[source,text]
+----
+set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
+set SOLR_SSL_KEY_STORE_PASSWORD=secret
+set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
+set SOLR_SSL_TRUST_STORE_PASSWORD=secret
+REM Require clients to authenticate
+set SOLR_SSL_NEED_CLIENT_AUTH=false
+REM Enable clients to authenticate (but not require)
+set SOLR_SSL_WANT_CLIENT_AUTH=false
+----
+
+[[EnablingSSL-RunSingleNodeSolrusingSSL]]
+=== Run Single Node Solr using SSL
+
+Start Solr using the command shown below; by default clients will not be required to authenticate:
+
+**nix command*
+
+[source,text]
+----
+bin/solr -p 8984
+----
+
+*Windows command*
+
+[source,text]
+----
+bin\solr.cmd -p 8984
+----
+
+[[EnablingSSL-SolrCloud]]
+== SolrCloud
+
+This section describes how to run a two-node SolrCloud cluster with no initial collections and a single-node external ZooKeeper. The commands below assume you have already created the keystore described above.
+
+[[EnablingSSL-ConfigureZooKeeper]]
+=== Configure ZooKeeper
+
+[NOTE]
+====
+
+ZooKeeper does not support encrypted communication with clients like Solr. There are several related JIRA tickets where SSL support is being planned/worked on: https://issues.apache.org/jira/browse/ZOOKEEPER-235[ZOOKEEPER-235]; https://issues.apache.org/jira/browse/ZOOKEEPER-236[ZOOKEEPER-236]; https://issues.apache.org/jira/browse/ZOOKEEPER-1000[ZOOKEEPER-1000]; and https://issues.apache.org/jira/browse/ZOOKEEPER-2120[ZOOKEEPER-2120].
+
+====
+
+Before you start any SolrCloud nodes, you must configure your solr cluster properties in ZooKeeper, so that Solr nodes know to communicate via SSL.
+
+This section assumes you have created and started a single-node external ZooKeeper on port 2181 on localhost - see <<setting-up-an-external-zookeeper-ensemble.adoc#setting-up-an-external-zookeeper-ensemble,Setting Up an External ZooKeeper Ensemble>>
+
+The `urlScheme` cluster-wide property needs to be set to `https` before any Solr node starts up. The example below uses the `zkcli` tool that comes with the binary Solr distribution to do this:
+
+**nix command*
+
+[source,text]
+----
+server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
+----
+
+*Windows command*
+
+[source,text]
+----
+server\scripts\cloud-scripts\zkcli.bat -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https
+----
+
+If you have set up your ZooKeeper cluster to use a <<taking-solr-to-production.adoc#TakingSolrtoProduction-ZooKeeperchroot,chroot�for Solr>>, make sure you use the correct `zkhost` string with `zkcli`, e.g. `-zkhost localhost:2181/solr`.
+
+[[EnablingSSL-RunSolrCloudwithSSL]]
+=== Run SolrCloud with SSL
+
+[[EnablingSSL-CreateSolrhomedirectoriesfortwonodes]]
+==== Create Solr home directories for two nodes
+
+Create two copies of the `server/solr/` directory which will serve as the Solr home directories for each of your two SolrCloud nodes:
+
+**nix commands*
+
+[source,text]
+----
+mkdir cloud
+cp -r server/solr cloud/node1
+cp -r server/solr cloud/node2
+----
+
+*Windows commands*
+
+[source,text]
+----
+mkdir cloud
+xcopy /E server\solr cloud\node1\
+xcopy /E server\solr cloud\node2\
+----
+
+[[EnablingSSL-StartthefirstSolrnode]]
+==== Start the first Solr node
+
+Next, start the first Solr node on port 8984. Be sure to stop the standalone server first if you started it when working through the previous section on this page.
+
+**nix command*
+
+[source,text]
+----
+bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984
+----
+
+*Windows command*
+
+[source,text]
+----
+bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984
+----
+
+Notice the use of the `-s` option to set the location of the Solr home directory for node1.
+
+If you created your SSL key without all DNS names/IP addresses on which Solr nodes will run, you can tell Solr to skip hostname verification for inter-Solr-node communications by setting the `solr.ssl.checkPeerName` system property to `false`:
+
+**nix command*
+
+[source,text]
+----
+bin/solr -cloud -s cloud/node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
+----
+
+*Windows command*
+
+[source,text]
+----
+bin\solr.cmd -cloud -s cloud\node1 -z localhost:2181 -p 8984 -Dsolr.ssl.checkPeerName=false
+----
+
+[[EnablingSSL-StartthesecondSolrnode]]
+==== Start the second Solr node
+
+Finally, start the second Solr node on port 7574 - again, to skip hostname verification, add `-Dsolr.ssl.checkPeerName=false`;
+
+**nix command*
+
+[source,text]
+----
+bin/solr -cloud -s cloud/node2 -z localhost:2181 -p 7574
+----
+
+*Windows command*
+
+[source,text]
+----
+bin\solr.cmd -cloud -s cloud\node2 -z localhost:2181 -p 7574
+----
+
+[[EnablingSSL-ExampleClientActions]]
+== Example Client Actions
+
+[IMPORTANT]
+====
+
+cURL on OS X Mavericks (10.9) has degraded SSL support. For more information and workarounds to allow 1-way SSL, see http://curl.haxx.se/mail/archive-2013-10/0036.html . cURL on OS X Yosemite (10.10) is improved - 2-way SSL is possible - see http://curl.haxx.se/mail/archive-2014-10/0053.html .
+
+The cURL commands in the following sections will not work with the system `curl` on OS X Yosemite (10.10). Instead, the certificate supplied with the `-E` param must be in PKCS12 format, and the file supplied with the `--cacert` param must contain only the CA certificate, and no key (see <<EnablingSSL-ConvertthecertificateandkeytoPEMformatforusewithcURL,above>> for instructions on creating this file):
+
+[source,bash]
+----
+curl -E solr-ssl.keystore.p12:secret --cacert solr-ssl.cacert.pem ...
+----
+
+====
+
+[NOTE]
+====
+
+If your operating system does not include cURL, you can download binaries here: http://curl.haxx.se/download.html
+
+====
+
+// OLD_CONFLUENCE_ID: EnablingSSL-CreateaSolrCloudcollectionusingbin/solr
+
+[[EnablingSSL-CreateaSolrCloudcollectionusingbin_solr]]
+=== Create a SolrCloud collection using `bin/solr`
+
+Create a 2-shard, replicationFactor=1 collection named mycollection using the default configset (data_driven_schema_configs):
+
+**nix command*
+
+[source,text]
+----
+bin/solr create -c mycollection -shards 2
+----
+
+*Windows command*
+
+[source,text]
+----
+bin\solr.cmd create -c mycollection -shards 2
+----
+
+The `create` action will pass the `SOLR_SSL_*` properties set in your include file to the SolrJ code used to create the collection.
+
+[[EnablingSSL-RetrieveSolrCloudclusterstatususingcURL]]
+=== Retrieve SolrCloud cluster status using cURL
+
+To get the resulting cluster status (again, if you have not enabled client authentication, remove the `-E solr-ssl.pem:secret` option):
+
+[source,bash]
+----
+curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/admin/collections?action=CLUSTERSTATUS&wt=json&indent=on"
+----
+
+You should get a response that looks like this:
+
+[source,json]
+----
+{
+ "responseHeader":{
+ "status":0,
+ "QTime":2041},
+ "cluster":{
+ "collections":{
+ "mycollection":{
+ "shards":{
+ "shard1":{
+ "range":"80000000-ffffffff",
+ "state":"active",
+ "replicas":{"core_node1":{
+ "state":"active",
+ "base_url":"https://127.0.0.1:8984/solr",
+ "core":"mycollection_shard1_replica1",
+ "node_name":"127.0.0.1:8984_solr",
+ "leader":"true"}}},
+ "shard2":{
+ "range":"0-7fffffff",
+ "state":"active",
+ "replicas":{"core_node2":{
+ "state":"active",
+ "base_url":"https://127.0.0.1:7574/solr",
+ "core":"mycollection_shard2_replica1",
+ "node_name":"127.0.0.1:7574_solr",
+ "leader":"true"}}}},
+ "maxShardsPerNode":"1",
+ "router":{"name":"compositeId"},
+ "replicationFactor":"1"}},
+ "properties":{"urlScheme":"https"}}}
+----
+
+[[EnablingSSL-Indexdocumentsusingpost.jar]]
+=== Index documents using `post.jar`
+
+Use `post.jar` to index some example documents to the SolrCloud collection created above:
+
+[source,text]
+----
+cd example/exampledocs
+java -Djavax.net.ssl.keyStorePassword=secret -Djavax.net.ssl.keyStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStore=../../server/etc/solr-ssl.keystore.jks -Djavax.net.ssl.trustStorePassword=secret -Durl=https://localhost:8984/solr/mycollection/update -jar post.jar *.xml
+----
+
+[[EnablingSSL-QueryusingcURL]]
+=== Query using cURL
+
+Use cURL to query the SolrCloud collection created above, from a directory containing the PEM formatted certificate and key created above (e.g. `example/etc/`) - if you have not enabled client authentication (system property `-Djetty.ssl.clientAuth=true)`, then you can remove the `-E solr-ssl.pem:secret` option:
+
+[source,bash]
+----
+curl -E solr-ssl.pem:secret --cacert solr-ssl.pem "https://localhost:8984/solr/mycollection/select?q=*:*&wt=json&indent=on"
+----
+
+[[EnablingSSL-IndexadocumentusingCloudSolrClient]]
+=== Index a document using `CloudSolrClient`
+
+From a java client using Solrj, index a document. In the code below, the `javax.net.ssl.*` system properties are set programmatically, but you could instead specify them on the java command line, as in the `post.jar` example above:
+
+[source,java]
+----
+System.setProperty("javax.net.ssl.keyStore", "/path/to/solr-ssl.keystore.jks");
+System.setProperty("javax.net.ssl.keyStorePassword", "secret");
+System.setProperty("javax.net.ssl.trustStore", "/path/to/solr-ssl.keystore.jks");
+System.setProperty("javax.net.ssl.trustStorePassword", "secret");
+String zkHost = "127.0.0.1:2181";
+CloudSolrClient client = new CloudSolrClient.Builder().withZkHost(zkHost).build();
+client.setDefaultCollection("mycollection");
+SolrInputDocument doc = new SolrInputDocument();
+doc.addField("id", "1234");
+doc.addField("name", "A lovely summer holiday");
+client.add(doc);
+client.commit();
+----
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/errata.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/errata.adoc b/solr/solr-ref-guide/src/errata.adoc
new file mode 100644
index 0000000..54a9dbb
--- /dev/null
+++ b/solr/solr-ref-guide/src/errata.adoc
@@ -0,0 +1,21 @@
+= Errata
+:page-shortname: errata
+:page-permalink: errata.html
+
+[[Errata-ErrataForThisDocumentation]]
+= Errata For This Documentation
+
+Any mistakes found in this documentation after its release will be listed on the on-line version of this page:
+
+http://s.apache.org/errata[https://cwiki.apache.org/confluence/display/solr/Errata]
+
+[[Errata-ErrataForPastVersionsofThisDocumentation]]
+= Errata For Past Versions of This Documentation
+
+Any known mistakes in past releases of this documentation will be noted below.
+
+*Using copyField directives with suggester:* Previous versions of this guide advocated using copyField directives to accumulate the contents on multiple fields into a single field to be used with Solr suggester components. This will not work previous to Solr 5.1; attempting to build the suggester will result in errors being reported in the logs if the field is multiValued. As a work-around, indexing clients should accumulate all of the contents into the field before sending the documents to Solr, and any fields used with the suggesters should have multiValued="false".
+
+The *_variable_ facet.range.gap parameter* was included in documentation even though the patch was not committed. As of yet there is no ability to specify variable gaps via a comma-separated list for facet.range.gap. Some of this functionality can be achieved by interval faceting, see SOLR-6216.
+
+The *MaxIndexingThreads* parameter in *solrconfig.xml* is no longer supported from Solr 5.3, see LUCENE-6659
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/45a148a7/solr/solr-ref-guide/src/exporting-result-sets.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/exporting-result-sets.adoc b/solr/solr-ref-guide/src/exporting-result-sets.adoc
new file mode 100644
index 0000000..5bbfc64
--- /dev/null
+++ b/solr/solr-ref-guide/src/exporting-result-sets.adoc
@@ -0,0 +1,54 @@
+= Exporting Result Sets
+:page-shortname: exporting-result-sets
+:page-permalink: exporting-result-sets.html
+
+It's possible to export fully sorted result sets using a special <<query-re-ranking.adoc#query-re-ranking,rank query parser>> and <<response-writers.adoc#response-writers,response writer>> specifically designed to work together to handle scenarios that involve sorting and exporting millions of records. This uses a stream sorting technique that begins to send records within milliseconds and continues to stream results until the entire result set has been sorted and exported.
+
+The cases where this functionality may be useful include: session analysis, distributed merge joins, time series roll-ups, aggregations on high cardinality fields, fully distributed field collapsing, and sort based stats.
+
+[[ExportingResultSets-FieldRequirements]]
+== Field Requirements
+
+All the fields being sorted and exported must have docValues set to true. For more information, see the section on <<docvalues.adoc#docvalues,DocValues>>.
+
+// OLD_CONFLUENCE_ID: ExportingResultSets-The/exportRequestHandler
+
+[[ExportingResultSets-The_exportRequestHandler]]
+== The `/export` RequestHandler
+
+The `/export` request handler with the appropriate configuration is one of Solr's out-of-the-box request handlers - see <<implicit-requesthandlers.adoc#implicit-requesthandlers,Implicit RequestHandlers>> for more information.
+
+Note that this request handler's properties are defined as "invariants", which means they cannot be overridden by other properties passed at another time (such as at query time).
+
+[[ExportingResultSets-RequestingResultsExport]]
+== Requesting Results Export
+
+You can use `/export` to make requests to export the result set of a query.
+
+All queries must include `sort` and `fl` parameters, or the query will return an error. Filter queries are also supported.
+
+The supported response writers are '`json`' and '`javabin`'. For backward compatibility reasons '`wt=xsort`' is also supported as input, but '`wt=xsort`' behaves same as '`wt=json`'. The default output format is `json`.
+
+Here is an example of what an export request of some indexed log data might look like:
+
+[source,java]
+----
+http://localhost:8983/solr/core_name/export?q=my-query&sort=severity+desc,timestamp+desc&fl=severity,timestamp,msg
+----
+
+[[ExportingResultSets-SpecifyingtheSortCriteria]]
+=== *Specifying the Sort Criteria*
+
+The `sort` property defines how documents will be sorted in the exported result set. Results can be sorted by any field that has a field type of int,long, float, double, string. The sort fields must be single valued fields.
+
+Up to four sort fields can be specified per request, with the 'asc' or 'desc' properties.
+
+[[ExportingResultSets-SpecifyingtheFieldList]]
+=== *Specifying the Field List*
+
+The `fl` property defines the fields that will be exported with the result set. Any of the field types that can be sorted (i.e., int, long, float, double, string, date, boolean) can be used in the field list. The fields can be single or multi-valued. However, returning scores and wildcards are not supported at this time.
+
+[[ExportingResultSets-DistributedSupport]]
+== *Distributed Support*
+
+See the section <<streaming-expressions.adoc#streaming-expressions,Streaming Expressions>> for distributed support.