You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by is...@apache.org on 2017/07/29 21:59:42 UTC
[05/28] lucene-solr:jira/solr-6630: Merging master
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8d00e53b/solr/solr-ref-guide/src/suggester.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/suggester.adoc b/solr/solr-ref-guide/src/suggester.adoc
index 61bf53d..18a7def 100644
--- a/solr/solr-ref-guide/src/suggester.adoc
+++ b/solr/solr-ref-guide/src/suggester.adoc
@@ -32,15 +32,11 @@ The main features of this Suggester are:
* Term dictionary pluggability, giving you the flexibility to choose the dictionary implementation
* Distributed support
-The `solrconfig.xml` found in Solr's "```techproducts```" example has the new Suggester implementation configured already. For more on search components, see the section <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,RequestHandlers and SearchComponents in SolrConfig>>.
-
-[[Suggester-ConfiguringSuggesterinsolrconfig.xml]]
-== Configuring Suggester in solrconfig.xml
+The `solrconfig.xml` found in Solr's "```techproducts```" example has a Suggester implementation configured already. For more on search components, see the section <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,RequestHandlers and SearchComponents in SolrConfig>>.
The "```techproducts```" example `solrconfig.xml` has a `suggest` search component and a `/suggest` request handler already configured. You can use that as the basis for your configuration, or create it from scratch, as detailed below.
-[[Suggester-AddingtheSuggestSearchComponent]]
-=== Adding the Suggest Search Component
+== Adding the Suggest Search Component
The first step is to add a search component to `solrconfig.xml` and tell it to use the SuggestComponent. Here is some sample code that could be used.
@@ -59,25 +55,33 @@ The first step is to add a search component to `solrconfig.xml` and tell it to u
</searchComponent>
----
-[[Suggester-SuggesterSearchComponentParameters]]
-==== Suggester Search Component Parameters
+=== Suggester Search Component Parameters
+
+The Suggester search component takes several configuration parameters.
+
+The choice of the lookup implementation (`lookupImpl`, how terms are found in the suggestion dictionary) and the dictionary implementation (`dictionaryImpl`, how terms are stored in the suggestion dictionary) will dictate some of the parameters required.
-The Suggester search component takes several configuration parameters. The choice of the lookup implementation (`lookupImpl`, how terms are found in the suggestion dictionary) and the dictionary implementation (`dictionaryImpl`, how terms are stored in the suggestion dictionary) will dictate some of the parameters required. Below are the main parameters that can be used no matter what lookup or dictionary implementation is used. In the following sections additional parameters are provided for each implementation.
+Below are the main parameters that can be used no matter what lookup or dictionary implementation is used. In the following sections additional parameters are provided for each implementation.
-// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
+`searchComponent name`::
+Arbitrary name for the search component.
-[cols="30,70",options="header"]
-|===
-|Parameter |Description
-|searchComponent name |Arbitrary name for the search component.
-|name |A symbolic name for this suggester. You can refer to this name in the URL parameters and in the SearchHandler configuration. It is possible to have mutiples of these
-|lookupImpl |Lookup implementation. There are several possible implementations, described below in the section <<Suggester-LookupImplementations,Lookup Implementations>>. If not set, the default lookup is JaspellLookupFactory.
-|dictionaryImpl |The dictionary implementation to use. There are several possible implementations, described below in the section <<Suggester-DictionaryImplementations,Dictionary Implementations>> . If not set, the default dictionary implementation is HighFrequencyDictionaryFactory unless a `sourceLocation` is used, in which case, the dictionary implementation will be FileDictionaryFactory
-|field a|
-A field from the index to use as the basis of suggestion terms. If `sourceLocation` is empty (meaning any dictionary implementation other than FileDictionaryFactory) then terms from this field in the index will be used.
+`name`::
+A symbolic name for this suggester. You can refer to this name in the URL parameters and in the SearchHandler configuration. It is possible to have multiples of these in one `solrconfig.xml` file.
-To be used as the basis for a suggestion, the field must be stored. You may want to <<copying-fields.adoc#copying-fields,use copyField rules>> to create a special 'suggest' field comprised of terms from other fields in documents. In any event, you likely want a minimal amount of analysis on the field, so an additional option is to create a field type in your schema that only uses basic tokenizers or filters. One option for such a field type is shown here:
+`lookupImpl`::
+Lookup implementation. There are several possible implementations, described below in the section <<Lookup Implementations>>. If not set, the default lookup is `JaspellLookupFactory`.
+`dictionaryImpl`::
+The dictionary implementation to use. There are several possible implementations, described below in the section <<Dictionary Implementations>>.
++
+If not set, the default dictionary implementation is `HighFrequencyDictionaryFactory`. However, if a `sourceLocation` is used, the dictionary implementation will be `FileDictionaryFactory`.
+
+`field`::
+A field from the index to use as the basis of suggestion terms. If `sourceLocation` is empty (meaning any dictionary implementation other than `FileDictionaryFactory`), then terms from this field in the index will be used.
++
+To be used as the basis for a suggestion, the field must be stored. You may want to <<copying-fields.adoc#copying-fields,use copyField rules>> to create a special 'suggest' field comprised of terms from other fields in documents. In any event, you very likely want a minimal amount of analysis on the field, so an additional option is to create a field type in your schema that only uses basic tokenizers or filters. One option for such a field type is shown here:
++
[source,xml]
----
<fieldType class="solr.TextField" name="textSuggest" positionIncrementGap="100">
@@ -88,154 +92,211 @@ To be used as the basis for a suggestion, the field must be stored. You may want
</analyzer>
</fieldType>
----
++
+However, this minimal analysis is not required if you want more analysis to occur on terms. If using the `AnalyzingLookupFactory` as your `lookupImpl`, however, you have the option of defining the field type rules to use for index and query time analysis.
+
+`sourceLocation`::
+The path to the dictionary file if using the `FileDictionaryFactory`. If this value is empty then the main index will be used as a source of terms and weights.
+
+`storeDir`::
+The location to store the dictionary file.
-However, this minimal analysis is not required if you want more analysis to occur on terms. If using the AnalyzingLookupFactory as your lookupImpl, however, you have the option of defining the field type rules to use for index and query time analysis.
+`buildOnCommit` and `buildOnOptimize`::
+If `true`, the lookup data structure will be rebuilt after soft-commit. If `false`, the default, then the lookup data will be built only when requested by URL parameter `suggest.build=true`. Use `buildOnCommit` to rebuild the dictionary with every soft-commit, or `buildOnOptimize` to build the dictionary only when the index is optimized.
++
+Some lookup implementations may take a long time to build, especially with large indexes. In such cases, using `buildOnCommit` or `buildOnOptimize`, particularly with a high frequency of softCommits is not recommended; it's recommended instead to build the suggester at a lower frequency by manually issuing requests with `suggest.build=true`.
-|sourceLocation |The path to the dictionary file if using the FileDictionaryFactory. If this value is empty then the main index will be used as a source of terms and weights.
-|storeDir |The location to store the dictionary file.
-|buildOnCommit or buildOnOptimize |If true then the lookup data structure will be rebuilt after soft-commit. If false, the default, then the lookup data will be built only when requested by URL parameter `suggest.build=true`. Use `buildOnCommit` to rebuild the dictionary with every soft-commit, or `buildOnOptimize` to build the dictionary only when the index is optimized. Some lookup implementations may take a long time to build, specially with large indexes, in such cases, using buildOnCommit or buildOnOptimize, particularly with a high frequency of softCommits is not recommended, and it's recommended instead to build the suggester at a lower frequency by manually issuing requests with `suggest.build=true`.
-|buildOnStartup |If true then the lookup data structure will be built when Solr starts or when the core is reloaded. If this parameter is not specified, the suggester will check if the lookup data structure is present on disk and build it if not found. Enabling this to true could lead to the core talking longer to load (or reload) as the suggester data structure needs to be built, which can sometimes take a long time. It’s usually preferred to have this setting set to 'false' and build suggesters manually issuing requests with `suggest.build=true`.
-|===
+`buildOnStartup`::
+If `true,` then the lookup data structure will be built when Solr starts or when the core is reloaded. If this parameter is not specified, the suggester will check if the lookup data structure is present on disk and build it if not found.
++
+Enabling this to `true` could lead to the core talking longer to load (or reload) as the suggester data structure needs to be built, which can sometimes take a long time. It’s usually preferred to have this setting set to `false`, the default, and build suggesters manually issuing requests with `suggest.build=true`.
-[[Suggester-LookupImplementations]]
-==== Lookup Implementations
+=== Lookup Implementations
The `lookupImpl` parameter defines the algorithms used to look up terms in the suggest index. There are several possible implementations to choose from, and some require additional parameters to be configured.
-[[Suggester-AnalyzingLookupFactory]]
-===== AnalyzingLookupFactory
+==== AnalyzingLookupFactory
A lookup that first analyzes the incoming text and adds the analyzed form to a weighted FST, and then does the same thing at lookup time.
This implementation uses the following additional properties:
-* suggestAnalyzerFieldType: The field type to use for the query-time and build-time term suggestion analysis.
-* exactMatchFirst: If true, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
-* preserveSep: If true, the default, then a separator between tokens is preserved. This means that suggestions are sensitive to tokenization (e.g., baseball is different from base ball).
-* preservePositionIncrements: If true, the suggester will preserve position increments. This means that token filters which leave gaps (for example, when StopFilter matches a stopword) the position would be respected when building the suggester. The default is false.
+`suggestAnalyzerFieldType`::
+The field type to use for the query-time and build-time term suggestion analysis.
-[[Suggester-FuzzyLookupFactory]]
-===== FuzzyLookupFactory
+`exactMatchFirst`::
+If `true`, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
+
+`preserveSep`::
+If `true`, the default, then a separator between tokens is preserved. This means that suggestions are sensitive to tokenization (e.g., baseball is different from base ball).
+
+`preservePositionIncrements`::
+If `true`, the suggester will preserve position increments. This means that token filters which leave gaps (for example, when StopFilter matches a stopword) the position would be respected when building the suggester. The default is `false`.
+
+==== FuzzyLookupFactory
This is a suggester which is an extension of the AnalyzingSuggester but is fuzzy in nature. The similarity is measured by the Levenshtein algorithm.
This implementation uses the following additional properties:
-* exactMatchFirst: If true, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
-* preserveSep: If true, the default, then a separator between tokens is preserved. This means that suggestions are sensitive to tokenization (e.g., baseball is different from base ball).
-* maxSurfaceFormsPerAnalyzedForm: Maximum number of surface forms to keep for a single analyzed form. When there are too many surface forms we discard the lowest weighted ones.
-* maxGraphExpansions: When building the FST ("index-time"), we add each path through the tokenstream graph as an individual entry. This places an upper-bound on how many expansions will be added for a single suggestion. The default is -1 which means there is no limit.
-* preservePositionIncrements: If true, the suggester will preserve position increments. This means that token filters which leave gaps (for example, when StopFilter matches a stopword) the position would be respected when building the suggester. The default is false.
-* maxEdits: The maximum number of string edits allowed. The systems hard limit is 2. The default is 1.
-* transpositions: If true, the default, transpositions should be treated as a primitive edit operation.
-* nonFuzzyPrefix: The length of the common non fuzzy prefix match which must match a suggestion. The default is 1.
-* minFuzzyLength: The minimum length of query before which any string edits will be allowed. The default is 3.
-* unicodeAware: If true, maxEdits, minFuzzyLength, transpositions and nonFuzzyPrefix parameters will be measured in unicode code points (actual letters) instead of bytes. The default is false.
+`exactMatchFirst`::
+If `true`, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
+
+`preserveSep`::
+If `true`, the default, then a separator between tokens is preserved. This means that suggestions are sensitive to tokenization (e.g., baseball is different from base ball).
+
+`maxSurfaceFormsPerAnalyzedForm`::
+The maximum number of surface forms to keep for a single analyzed form. When there are too many surface forms we discard the lowest weighted ones.
-[[Suggester-AnalyzingInfixLookupFactory]]
-===== AnalyzingInfixLookupFactory
+`maxGraphExpansions`::
+When building the FST ("index-time"), we add each path through the tokenstream graph as an individual entry. This places an upper-bound on how many expansions will be added for a single suggestion. The default is `-1` which means there is no limit.
+
+`preservePositionIncrements`::
+If `true`, the suggester will preserve position increments. This means that token filters which leave gaps (for example, when StopFilter matches a stopword) the position would be respected when building the suggester. The default is `false`.
+
+`maxEdits`::
+The maximum number of string edits allowed. The system's hard limit is `2`. The default is `1`.
+
+`transpositions`::
+If `true`, the default, transpositions should be treated as a primitive edit operation.
+
+`nonFuzzyPrefix`::
+The length of the common non fuzzy prefix match which must match a suggestion. The default is `1`.
+
+`minFuzzyLength`::
+The minimum length of query before which any string edits will be allowed. The default is `3`.
+
+`unicodeAware`::
+If `true`, the `maxEdits`, `minFuzzyLength`, `transpositions` and `nonFuzzyPrefix` parameters will be measured in unicode code points (actual letters) instead of bytes. The default is `false`.
+
+==== AnalyzingInfixLookupFactory
Analyzes the input text and then suggests matches based on prefix matches to any tokens in the indexed text. This uses a Lucene index for its dictionary.
This implementation uses the following additional properties.
-* indexPath: When using AnalyzingInfixSuggester you can provide your own path where the index will get built. The default is analyzingInfixSuggesterIndexDir and will be created in your collections data directory.
-* minPrefixChars: Minimum number of leading characters before PrefixQuery is used (default is 4). Prefixes shorter than this are indexed as character ngrams (increasing index size but making lookups faster).
-* allTermsRequired: Boolean option for multiple terms. Default is true - all terms required.
-* highlight: Highlight suggest terms. Default is true.
+`indexPath`::
+When using `AnalyzingInfixSuggester` you can provide your own path where the index will get built. The default is `analyzingInfixSuggesterIndexDir` and will be created in your collection's `data/` directory.
+
+`minPrefixChars`::
+Minimum number of leading characters before PrefixQuery is used (default is `4`). Prefixes shorter than this are indexed as character ngrams (increasing index size but making lookups faster).
+
+`allTermsRequired`::
+Boolean option for multiple terms. The default is `true`, all terms will be required.
+
+`highlight`::
+Highlight suggest terms. Default is `true`.
-This implementation supports <<Suggester-ContextFiltering,Context Filtering>>.
+This implementation supports <<Context Filtering>>.
-[[Suggester-BlendedInfixLookupFactory]]
-===== BlendedInfixLookupFactory
+==== BlendedInfixLookupFactory
-An extension of the AnalyzingInfixSuggester which provides additional functionality to weight prefix matches across the matched documents. You can tell it to score higher if a hit is closer to the start of the suggestion or vice versa.
+An extension of the `AnalyzingInfixSuggester` which provides additional functionality to weight prefix matches across the matched documents. You can tell it to score higher if a hit is closer to the start of the suggestion or vice versa.
This implementation uses the following additional properties:
-* blenderType: used to calculate weight coefficient using the position of the first matching word. Can be one of:
-** position_linear: weightFieldValue*(1 - 0.10*position): Matches to the start will be given a higher score (Default)
-** position_reciprocal: weightFieldValue/(1+position): Matches to the end will be given a higher score.
-*** exponent: an optional configuration variable for the position_reciprocal blenderType used to control how fast the score will increase or decrease. Default 2.0.
-* numFactor: The factor to multiply the number of searched elements from which results will be pruned. Default is 10.
-* indexPath: When using BlendedInfixSuggester you can provide your own path where the index will get built. The default directory name is blendedInfixSuggesterIndexDir and will be created in your collections data directory.
-* minPrefixChars: Minimum number of leading characters before PrefixQuery is used (default 4). Prefixes shorter than this are indexed as character ngrams (increasing index size but making lookups faster).
+`blenderType`::
+Used to calculate weight coefficient using the position of the first matching word. Can be one of:
+`position_linear`:::
+`weightFieldValue*(1 - 0.10*position)`: Matches to the start will be given a higher score. This is the default.
+`position_reciprocal`:::
+`weightFieldValue/(1+position)`: Matches to the end will be given a higher score.
+`exponent`::::
+An optional configuration variable for the position_reciprocal blenderType used to control how fast the score will increase or decrease. Default `2.0`.
-This implementation supports <<Suggester-ContextFiltering,Context Filtering>> .
+`numFactor`::
+The factor to multiply the number of searched elements from which results will be pruned. Default is `10`.
-[[Suggester-FreeTextLookupFactory]]
-===== FreeTextLookupFactory
+`indexPath`::
+When using `BlendedInfixSuggester` you can provide your own path where the index will get built. The default directory name is `blendedInfixSuggesterIndexDir` and will be created in your collections data directory.
+
+`minPrefixChars`::
+Minimum number of leading characters before PrefixQuery is used (the default is `4`). Prefixes shorter than this are indexed as character ngrams (increasing index size but making lookups faster).
+
+This implementation supports <<Context Filtering>> .
+
+==== FreeTextLookupFactory
It looks at the last tokens plus the prefix of whatever final token the user is typing, if present, to predict the most likely next token. The number of previous tokens that need to be considered can also be specified. This suggester would only be used as a fallback, when the primary suggester fails to find any suggestions.
This implementation uses the following additional properties:
-* suggestFreeTextAnalyzerFieldType: The analyzer used at "query-time" and "build-time" to analyze suggestions. This field is required.
-* ngrams: The max number of tokens out of which singles will be make the dictionary. The default value is 2. Increasing this would mean you want more than the previous 2 tokens to be taken into consideration when making the suggestions.
+`suggestFreeTextAnalyzerFieldType`::
+The analyzer used at "query-time" and "build-time" to analyze suggestions. This parameter is required.
-[[Suggester-FSTLookupFactory]]
-===== FSTLookupFactory
+`ngrams`::
+The max number of tokens out of which singles will be made the dictionary. The default value is `2`. Increasing this would mean you want more than the previous 2 tokens to be taken into consideration when making the suggestions.
+
+==== FSTLookupFactory
An automaton-based lookup. This implementation is slower to build, but provides the lowest memory cost. We recommend using this implementation unless you need more sophisticated matching results, in which case you should use the Jaspell implementation.
This implementation uses the following additional properties:
-* exactMatchFirst: If true, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
-* weightBuckets: The number of separate buckets for weights which the suggester will use while building its dictionary.
+`exactMatchFirst`::
+If `true`, the default, exact suggestions are returned first, even if they are prefixes or other strings in the FST have larger weights.
+
+`weightBuckets`::
+The number of separate buckets for weights which the suggester will use while building its dictionary.
-[[Suggester-TSTLookupFactory]]
-===== TSTLookupFactory
+==== TSTLookupFactory
A simple compact ternary trie based lookup.
-[[Suggester-WFSTLookupFactory]]
-===== WFSTLookupFactory
+==== WFSTLookupFactory
+
+A weighted automaton representation which is an alternative to `FSTLookup` for more fine-grained ranking. `WFSTLookup` does not use buckets, but instead a shortest path algorithm.
-A weighted automaton representation which is an alternative to FSTLookup for more fine-grained ranking. WFSTLookup does not use buckets, but instead a shortest path algorithm. Note that it expects weights to be whole numbers. If weight is missing it's assumed to be 1.0. Weights affect the sorting of matching suggestions when `spellcheck.onlyMorePopular=true` is selected: weights are treated as "popularity" score, with higher weights preferred over suggestions with lower weights.
+Note that it expects weights to be whole numbers. If weight is missing it's assumed to be `1.0`. Weights affect the sorting of matching suggestions when `spellcheck.onlyMorePopular=true` is selected: weights are treated as "popularity" score, with higher weights preferred over suggestions with lower weights.
-[[Suggester-JaspellLookupFactory]]
-===== JaspellLookupFactory
+==== JaspellLookupFactory
A more complex lookup based on a ternary trie from the http://jaspell.sourceforge.net/[JaSpell] project. Use this implementation if you need more sophisticated matching results.
-[[Suggester-DictionaryImplementations]]
-==== Dictionary Implementations
+=== Dictionary Implementations
The dictionary implementations define how terms are stored. There are several options, and multiple dictionaries can be used in a single request if necessary.
-[[Suggester-DocumentDictionaryFactory]]
-===== DocumentDictionaryFactory
+==== DocumentDictionaryFactory
A dictionary with terms, weights, and an optional payload taken from the index.
This dictionary implementation takes the following parameters in addition to parameters described for the Suggester generally and for the lookup implementation:
-* weightField: A field that is stored or a numeric DocValue field. This field is optional.
-* payloadField: The payloadField should be a field that is stored. This field is optional.
-* contextField: Field to be used for context filtering. Note that only some lookup implementations support filtering.
+`weightField`::
+A field that is stored or a numeric DocValue field. This parameter is optional.
-[[Suggester-DocumentExpressionDictionaryFactory]]
-===== DocumentExpressionDictionaryFactory
+`payloadField`::
+The `payloadField` should be a field that is stored. This parameter is optional.
-This dictionary implementation is the same as the DocumentDictionaryFactory but allows users to specify an arbitrary expression into the 'weightExpression' tag.
+`contextField`::
+Field to be used for context filtering. Note that only some lookup implementations support filtering.
+
+==== DocumentExpressionDictionaryFactory
+
+This dictionary implementation is the same as the `DocumentDictionaryFactory` but allows users to specify an arbitrary expression into the `weightExpression` tag.
This dictionary implementation takes the following parameters in addition to parameters described for the Suggester generally and for the lookup implementation:
-* payloadField: The payloadField should be a field that is stored. This field is optional.
-* weightExpression: An arbitrary expression used for scoring the suggestions. The fields used must be numeric fields. This field is required.
-* contextField: Field to be used for context filtering. Note that only some lookup implementations support filtering.
+`payloadField`::
+The `payloadField` should be a field that is stored. This parameter is optional.
+
+`weightExpression`::
+An arbitrary expression used for scoring the suggestions. The fields used must be numeric fields. This parameter is required.
-[[Suggester-HighFrequencyDictionaryFactory]]
-===== HighFrequencyDictionaryFactory
+`contextField`::
+Field to be used for context filtering. Note that only some lookup implementations support filtering.
+
+==== HighFrequencyDictionaryFactory
This dictionary implementation allows adding a threshold to prune out less frequent terms in cases where very common terms may overwhelm other terms.
This dictionary implementation takes one parameter in addition to parameters described for the Suggester generally and for the lookup implementation:
-* threshold: A value between zero and one representing the minimum fraction of the total documents where a term should appear in order to be added to the lookup dictionary.
+`threshold`::
+A value between zero and one representing the minimum fraction of the total documents where a term should appear in order to be added to the lookup dictionary.
-[[Suggester-FileDictionaryFactory]]
-===== FileDictionaryFactory
+==== FileDictionaryFactory
This dictionary implementation allows using an external file that contains suggest entries. Weights and payloads can also be used.
@@ -243,8 +304,9 @@ If using a dictionary file, it should be a plain text file in UTF-8 encoding. Yo
This dictionary implementation takes one parameter in addition to parameters described for the Suggester generally and for the lookup implementation:
-fieldDelimiter:: Specify the delimiter to be used separating the entries, weights and payloads. The default is tab ('\t').
-
+`fieldDelimiter`::
+Specifies the delimiter to be used separating the entries, weights and payloads. The default is tab (`\t`).
++
.Example File
[source,text]
----
@@ -253,8 +315,7 @@ accidentally 2.0
accommodate 3.0
----
-[[Suggester-MultipleDictionaries]]
-==== Multiple Dictionaries
+=== Multiple Dictionaries
It is possible to include multiple `dictionaryImpl` definitions in a single SuggestComponent definition.
@@ -285,10 +346,9 @@ To do this, simply define separate suggesters, as in this example:
</searchComponent>
----
-When using these Suggesters in a query, you would define multiple 'suggest.dictionary' parameters in the request, referring to the names given for each Suggester in the search component definition. The response will include the terms in sections for each Suggester. See the <<Suggester-ExampleUsages,Examples>> section below for an example request and response.
+When using these Suggesters in a query, you would define multiple `suggest.dictionary` parameters in the request, referring to the names given for each Suggester in the search component definition. The response will include the terms in sections for each Suggester. See the <<Example Usages>> section below for an example request and response.
-[[Suggester-AddingtheSuggestRequestHandler]]
-=== Adding the Suggest Request Handler
+== Adding the Suggest Request Handler
After adding the search component, a request handler must be added to `solrconfig.xml`. This request handler works the <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,same as any other request handler>>, and allows you to configure default parameters for serving suggestion requests. The request handler definition must incorporate the "suggest" search component defined previously.
@@ -305,51 +365,59 @@ After adding the search component, a request handler must be added to `solrconfi
</requestHandler>
----
-[[Suggester-SuggestRequestHandlerParameters]]
-==== Suggest Request Handler Parameters
+=== Suggest Request Handler Parameters
The following parameters allow you to set defaults for the Suggest request handler:
-// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
-
-[cols="30,70",options="header"]
-|===
-|Parameter |Description
-|suggest=true |This parameter should always be true, because we always want to run the Suggester for queries submitted to this handler.
-|suggest.dictionary |The name of the dictionary component configured in the search component. This is a mandatory parameter. It can be set in the request handler, or sent as a parameter at query time.
-|suggest.q |The query to use for suggestion lookups.
-|suggest.count |Specifies the number of suggestions for Solr to return.
-|suggest.cfq |A Context Filter Query used to filter suggestions based on the context field, if supported by the suggester.
-|suggest.build |If true, it will build the suggester index. This is likely useful only for initial requests; you would probably not want to build the dictionary on every request, particularly in a production system. If you would like to keep your dictionary up to date, you should use the `buildOnCommit` or `buildOnOptimize` parameter for the search component.
-|suggest.reload |If true, it will reload the suggester index.
-|suggest.buildAll |If true, it will build all suggester indexes.
-|suggest.reloadAll |If true, it will reload all suggester indexes.
-|===
+`suggest=true`::
+This parameter should always be `true`, because we always want to run the Suggester for queries submitted to this handler.
+
+`suggest.dictionary`::
+The name of the dictionary component configured in the search component. This is a mandatory parameter. It can be set in the request handler, or sent as a parameter at query time.
+
+`suggest.q`::
+The query to use for suggestion lookups.
+
+`suggest.count`::
+Specifies the number of suggestions for Solr to return.
+
+`suggest.cfq`::
+A Context Filter Query used to filter suggestions based on the context field, if supported by the suggester.
+
+`suggest.build`::
+If `true`, it will build the suggester index. This is likely useful only for initial requests; you would probably not want to build the dictionary on every request, particularly in a production system. If you would like to keep your dictionary up to date, you should use the `buildOnCommit` or `buildOnOptimize` parameter for the search component.
+
+`suggest.reload`::
+If `true`, it will reload the suggester index.
+
+`suggest.buildAll`::
+If `true`, it will build all suggester indexes.
+
+`suggest.reloadAll`::
+If `true`, it will reload all suggester indexes.
These properties can also be overridden at query time, or not set in the request handler at all and always sent at query time.
.Context Filtering
[IMPORTANT]
====
-Context filtering (`suggest.cfq`) is currently only supported by AnalyzingInfixLookupFactory and BlendedInfixLookupFactory, and only when backed by a Document*Dictionary. All other implementations will return unfiltered matches as if filtering was not requested.
+Context filtering (`suggest.cfq`) is currently only supported by `AnalyzingInfixLookupFactory` and `BlendedInfixLookupFactory`, and only when backed by a `Document*Dictionary`. All other implementations will return unfiltered matches as if filtering was not requested.
====
-[[Suggester-ExampleUsages]]
== Example Usages
-[[Suggester-GetSuggestionswithWeights]]
=== Get Suggestions with Weights
-This is the basic suggestion using a single dictionary and a single Solr core.
+This is a basic suggestion using a single dictionary and a single Solr core.
Example query:
[source,text]
----
-http://localhost:8983/solr/techproducts/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&wt=json&suggest.q=elec
+http://localhost:8983/solr/techproducts/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&suggest.q=elec
----
-In this example, we've simply requested the string 'elec' with the suggest.q parameter and requested that the suggestion dictionary be built with suggest.build (note, however, that you would likely not want to build the index on every query - instead you should use buildOnCommit or buildOnOptimize if you have regularly changing documents).
+In this example, we've simply requested the string 'elec' with the `suggest.q` parameter and requested that the suggestion dictionary be built with `suggest.build` (note, however, that you would likely not want to build the index on every query - instead you should use `buildOnCommit` or `buildOnOptimize` if you have regularly changing documents).
Example response:
@@ -388,8 +456,7 @@ Example response:
}
----
-[[Suggester-MultipleDictionaries.1]]
-=== Multiple Dictionaries
+=== Using Multiple Dictionaries
If you have defined multiple dictionaries, you can use them in queries.
@@ -397,11 +464,10 @@ Example query:
[source,text]
----
-http://localhost:8983/solr/techproducts/suggest?suggest=true& \
- suggest.dictionary=mySuggester&suggest.dictionary=altSuggester&wt=json&suggest.q=elec
+http://localhost:8983/solr/techproducts/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.dictionary=altSuggester&suggest.q=elec
----
-In this example we have sent the string 'elec' as the suggest.q parameter and named two suggest.dictionary definitions to be used.
+In this example we have sent the string 'elec' as the `suggest.q` parameter and named two `suggest.dictionary` definitions to be used.
Example response:
@@ -441,10 +507,9 @@ Example response:
}
----
-[[Suggester-ContextFiltering]]
=== Context Filtering
-Context filtering lets you filter suggestions by a separate context field, such as category, department or any other token. The AnalyzingInfixLookupFactory and BlendedInfixLookupFactory currently support this feature, when backed by DocumentDictionaryFactory.
+Context filtering lets you filter suggestions by a separate context field, such as category, department or any other token. The `AnalyzingInfixLookupFactory` and `BlendedInfixLookupFactory` currently support this feature, when backed by `DocumentDictionaryFactory`.
Add `contextField` to your suggester configuration. This example will suggest names and allow to filter by category:
@@ -469,8 +534,7 @@ Example context filtering suggest query:
[source,text]
----
-http://localhost:8983/solr/techproducts/suggest?suggest=true&suggest.build=true& \
- suggest.dictionary=mySuggester&wt=json&suggest.q=c&suggest.cfq=memory
+http://localhost:8983/solr/techproducts/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&suggest.q=c&suggest.cfq=memory
----
-The suggester will only bring back suggestions for products tagged with cat=memory.
+The suggester will only bring back suggestions for products tagged with 'cat=memory'.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8d00e53b/solr/solr-ref-guide/src/taking-solr-to-production.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/taking-solr-to-production.adoc b/solr/solr-ref-guide/src/taking-solr-to-production.adoc
index 9763410..fe7ed08 100644
--- a/solr/solr-ref-guide/src/taking-solr-to-production.adoc
+++ b/solr/solr-ref-guide/src/taking-solr-to-production.adoc
@@ -20,17 +20,14 @@
This section provides guidance on how to setup Solr to run in production on *nix platforms, such as Ubuntu. Specifically, we’ll walk through the process of setting up to run a single Solr instance on a Linux host and then provide tips on how to support multiple Solr nodes running on the same host.
-[[TakingSolrtoProduction-ServiceInstallationScript]]
== Service Installation Script
Solr includes a service installation script (`bin/install_solr_service.sh`) to help you install Solr as a service on Linux. Currently, the script only supports CentOS, Debian, Red Hat, SUSE and Ubuntu Linux distributions. Before running the script, you need to determine a few parameters about your setup. Specifically, you need to decide where to install Solr and which system user should be the owner of the Solr files and process.
-[[TakingSolrtoProduction-Planningyourdirectorystructure]]
=== Planning Your Directory Structure
We recommend separating your live Solr files, such as logs and index files, from the files included in the Solr distribution bundle, as that makes it easier to upgrade Solr and is considered a good practice to follow as a system administrator.
-[[TakingSolrtoProduction-SolrInstallationDirectory]]
==== Solr Installation Directory
By default, the service installation script will extract the distribution archive into `/opt`. You can change this location using the `-i` option when running the installation script. The script will also create a symbolic link to the versioned directory of Solr. For instance, if you run the installation script for Solr {solr-docs-version}.0, then the following directory structure will be used:
@@ -43,19 +40,16 @@ By default, the service installation script will extract the distribution archiv
Using a symbolic link insulates any scripts from being dependent on the specific Solr version. If, down the road, you need to upgrade to a later version of Solr, you can just update the symbolic link to point to the upgraded version of Solr. We’ll use `/opt/solr` to refer to the Solr installation directory in the remaining sections of this page.
-[[TakingSolrtoProduction-SeparateDirectoryforWritableFiles]]
==== Separate Directory for Writable Files
You should also separate writable Solr files into a different directory; by default, the installation script uses `/var/solr`, but you can override this location using the `-d` option. With this approach, the files in `/opt/solr` will remain untouched and all files that change while Solr is running will live under `/var/solr`.
-[[TakingSolrtoProduction-CreatetheSolruser]]
=== Create the Solr User
Running Solr as `root` is not recommended for security reasons, and the <<solr-control-script-reference.adoc#solr-control-script-reference,control script>> start command will refuse to do so. Consequently, you should determine the username of a system user that will own all of the Solr files and the running Solr process. By default, the installation script will create the *solr* user, but you can override this setting using the -u option. If your organization has specific requirements for creating new user accounts, then you should create the user before running the script. The installation script will make the Solr user the owner of the `/opt/solr` and `/var/solr` directories.
You are now ready to run the installation script.
-[[TakingSolrtoProduction-RuntheSolrInstallationScript]]
=== Run the Solr Installation Script
To run the script, you'll need to download the latest Solr distribution archive and then do the following:
@@ -97,12 +91,10 @@ If you do not want to start the service immediately, pass the `-n` option. You c
We'll cover some additional configuration settings you can make to fine-tune your Solr setup in a moment. Before moving on, let's take a closer look at the steps performed by the installation script. This gives you a better overview and will help you understand important details about your Solr installation when reading other pages in this guide; such as when a page refers to Solr home, you'll know exactly where that is on your system.
-[[TakingSolrtoProduction-SolrHomeDirectory]]
==== Solr Home Directory
The Solr home directory (not to be confused with the Solr installation directory) is where Solr manages core directories with index files. By default, the installation script uses `/var/solr/data`. If the `-d` option is used on the install script, then this will change to the `data` subdirectory in the location given to the -d option. Take a moment to inspect the contents of the Solr home directory on your system. If you do not <<using-zookeeper-to-manage-configuration-files.adoc#using-zookeeper-to-manage-configuration-files,store `solr.xml` in ZooKeeper>>, the home directory must contain a `solr.xml` file. When Solr starts up, the Solr Control Script passes the location of the home directory using the `-Dsolr.solr.home=...` system property.
-[[TakingSolrtoProduction-Environmentoverridesincludefile]]
==== Environment Overrides Include File
The service installation script creates an environment specific include file that overrides defaults used by the `bin/solr` script. The main advantage of using an include file is that it provides a single location where all of your environment-specific overrides are defined. Take a moment to inspect the contents of the `/etc/default/solr.in.sh` file, which is the default path setup by the installation script. If you used the `-s` option on the install script to change the name of the service, then the first part of the filename will be different. For a service named `solr-demo`, the file will be named `/etc/default/solr-demo.in.sh`. There are many settings that you can override using this file. However, at a minimum, this script needs to define the `SOLR_PID_DIR` and `SOLR_HOME` variables, such as:
@@ -115,7 +107,6 @@ SOLR_HOME=/var/solr/data
The `SOLR_PID_DIR` variable sets the directory where the <<solr-control-script-reference.adoc#solr-control-script-reference,control script>> will write out a file containing the Solr server’s process ID.
-[[TakingSolrtoProduction-Logsettings]]
==== Log Settings
Solr uses Apache Log4J for logging. The installation script copies `/opt/solr/server/resources/log4j.properties` to `/var/solr/log4j.properties`. Take a moment to verify that the Solr include file is configured to send logs to the correct location by checking the following settings in `/etc/default/solr.in.sh`:
@@ -128,7 +119,6 @@ SOLR_LOGS_DIR=/var/solr/logs
For more information about Log4J configuration, please see: <<configuring-logging.adoc#configuring-logging,Configuring Logging>>
-[[TakingSolrtoProduction-init.dscript]]
==== init.d Script
When running a service like Solr on Linux, it’s common to setup an init.d script so that system administrators can control Solr using the service tool, such as: `service solr start`. The installation script creates a very basic init.d script to help you get started. Take a moment to inspect the `/etc/init.d/solr` file, which is the default script name setup by the installation script. If you used the `-s` option on the install script to change the name of the service, then the filename will be different. Notice that the following variables are setup for your environment based on the parameters passed to the installation script:
@@ -149,7 +139,6 @@ service solr start
The `/etc/init.d/solr` script also supports the **stop**, **restart**, and *status* commands. Please keep in mind that the init script that ships with Solr is very basic and is intended to show you how to setup Solr as a service. However, it’s also common to use more advanced tools like *supervisord* or *upstart* to control Solr as a service on Linux. While showing how to integrate Solr with tools like supervisord is beyond the scope of this guide, the `init.d/solr` script should provide enough guidance to help you get started. Also, the installation script sets the Solr service to start automatically when the host machine initializes.
-[[TakingSolrtoProduction-ProgressCheck]]
=== Progress Check
In the next section, we cover some additional environment settings to help you fine-tune your production setup. However, before we move on, let's review what we've achieved thus far. Specifically, you should be able to control Solr using `/etc/init.d/solr`. Please verify the following commands work with your setup:
@@ -174,10 +163,8 @@ Solr process PID running on port 8983
If the `status` command is not successful, look for error messages in `/var/solr/logs/solr.log`.
-[[TakingSolrtoProduction-Finetuneyourproductionsetup]]
== Fine-Tune Your Production Setup
-[[TakingSolrtoProduction-MemoryandGCSettings]]
=== Memory and GC Settings
By default, the `bin/solr` script sets the maximum Java heap size to 512M (-Xmx512m), which is fine for getting started with Solr. For production, you’ll want to increase the maximum heap size based on the memory requirements of your search application; values between 10 and 20 gigabytes are not uncommon for production servers. When you need to change the memory settings for your Solr server, use the `SOLR_JAVA_MEM` variable in the include file, such as:
@@ -189,13 +176,11 @@ SOLR_JAVA_MEM="-Xms10g -Xmx10g"
Also, the <<solr-control-script-reference.adoc#solr-control-script-reference,Solr Control Script>> comes with a set of pre-configured Java Garbage Collection settings that have shown to work well with Solr for a number of different workloads. However, these settings may not work well for your specific use of Solr. Consequently, you may need to change the GC settings, which should also be done with the `GC_TUNE` variable in the `/etc/default/solr.in.sh` include file. For more information about tuning your memory and garbage collection settings, see: <<jvm-settings.adoc#jvm-settings,JVM Settings>>.
-[[TakingSolrtoProduction-Out-of-MemoryShutdownHook]]
==== Out-of-Memory Shutdown Hook
The `bin/solr` script registers the `bin/oom_solr.sh` script to be called by the JVM if an OutOfMemoryError occurs. The `oom_solr.sh` script will issue a `kill -9` to the Solr process that experiences the `OutOfMemoryError`. This behavior is recommended when running in SolrCloud mode so that ZooKeeper is immediately notified that a node has experienced a non-recoverable error. Take a moment to inspect the contents of the `/opt/solr/bin/oom_solr.sh` script so that you are familiar with the actions the script will perform if it is invoked by the JVM.
-[[TakingSolrtoProduction-SolrCloud]]
-=== SolrCloud
+=== Going to Production with SolrCloud
To run Solr in SolrCloud mode, you need to set the `ZK_HOST` variable in the include file to point to your ZooKeeper ensemble. Running the embedded ZooKeeper is not supported in production environments. For instance, if you have a ZooKeeper ensemble hosted on the following three hosts on the default client port 2181 (zk1, zk2, and zk3), then you would set:
@@ -206,7 +191,6 @@ ZK_HOST=zk1,zk2,zk3
When the `ZK_HOST` variable is set, Solr will launch in "cloud" mode.
-[[TakingSolrtoProduction-ZooKeeperchroot]]
==== ZooKeeper chroot
If you're using a ZooKeeper instance that is shared by other systems, it's recommended to isolate the SolrCloud znode tree using ZooKeeper's chroot support. For instance, to ensure all znodes created by SolrCloud are stored under `/solr`, you can put `/solr` on the end of your `ZK_HOST` connection string, such as:
@@ -225,12 +209,9 @@ bin/solr zk mkroot /solr -z <ZK_node>:<ZK_PORT>
[NOTE]
====
-
If you also want to bootstrap ZooKeeper with existing `solr_home`, you can instead use the `zkcli.sh` / `zkcli.bat` `bootstrap` command, which will also create the chroot path if it does not exist. See <<command-line-utilities.adoc#command-line-utilities,Command Line Utilities>> for more info.
-
====
-[[TakingSolrtoProduction-SolrHostname]]
=== Solr Hostname
Use the `SOLR_HOST` variable in the include file to set the hostname of the Solr server.
@@ -242,7 +223,6 @@ SOLR_HOST=solr1.example.com
Setting the hostname of the Solr server is recommended, especially when running in SolrCloud mode, as this determines the address of the node when it registers with ZooKeeper.
-[[TakingSolrtoProduction-Overridesettingsinsolrconfig.xml]]
=== Override Settings in solrconfig.xml
Solr allows configuration properties to be overridden using Java system properties passed at startup using the `-Dproperty=value` syntax. For instance, in `solrconfig.xml`, the default auto soft commit settings are set to:
@@ -268,7 +248,6 @@ The `bin/solr` script simply passes options starting with `-D` on to the JVM dur
SOLR_OPTS="$SOLR_OPTS -Dsolr.autoSoftCommit.maxTime=10000"
----
-[[TakingSolrtoProduction-RunningmultipleSolrnodesperhost]]
== Running Multiple Solr Nodes Per Host
The `bin/solr` script is capable of running multiple instances on one machine, but for a *typical* installation, this is not a recommended setup. Extra CPU and memory resources are required for each additional instance. A single instance is easily capable of handling multiple indexes.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8d00e53b/solr/solr-ref-guide/src/the-dismax-query-parser.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/the-dismax-query-parser.adoc b/solr/solr-ref-guide/src/the-dismax-query-parser.adoc
index 378fd93..1cb3014 100644
--- a/solr/solr-ref-guide/src/the-dismax-query-parser.adoc
+++ b/solr/solr-ref-guide/src/the-dismax-query-parser.adoc
@@ -33,50 +33,25 @@ ____
Whether or not you remember this explanation, do remember that the DisMax Query Parser was primarily designed to be easy to use and to accept almost any input without returning an error.
-[[TheDisMaxQueryParser-DisMaxParameters]]
-== DisMax Parameters
+== DisMax Query Parser Parameters
-In addition to the common request parameter, highlighting parameters, and simple facet parameters, the DisMax query parser supports the parameters described below. Like the standard query parser, the DisMax query parser allows default parameter values to be specified in `solrconfig.xml`, or overridden by query-time values in the request.
-
-// TODO: Change column width to %autowidth.spread when https://github.com/asciidoctor/asciidoctor-pdf/issues/599 is fixed
-
-[cols="30,70",options="header"]
-|===
-|Parameter |Description
-|<<TheDisMaxQueryParser-TheqParameter,q>> |Defines the raw input strings for the query.
-|<<TheDisMaxQueryParser-Theq.altParameter,q.alt>> |Calls the standard query parser and defines query input strings, when the q parameter is not used.
-|<<TheDisMaxQueryParser-Theqf_QueryFields_Parameter,qf>> |Query Fields: specifies the fields in the index on which to perform the query. If absent, defaults to `df`.
-|<<TheDisMaxQueryParser-Themm_MinimumShouldMatch_Parameter,mm>> |Minimum "Should" Match: specifies a minimum number of clauses that must match in a query. If no 'mm' parameter is specified in the query, or as a default in `solrconfig.xml`, the effective value of the `q.op` parameter (either in the query or as a default in `solrconfig.xml`) is used to influence the behavior. If `q.op` is effectively AND'ed, then mm=100%; if `q.op` is OR'ed, then mm=1. Users who want to force the legacy behavior should set a default value for the 'mm' parameter in their `solrconfig.xml` file. Users should add this as a configured default for their request handlers. This parameter tolerates miscellaneous white spaces in expressions (e.g., `" 3 < -25% 10 < -3\n", " \n-25%\n ", " \n3\n "`).
-|<<TheDisMaxQueryParser-Thepf_PhraseFields_Parameter,pf>> |Phrase Fields: boosts the score of documents in cases where all of the terms in the q parameter appear in close proximity.
-|<<TheDisMaxQueryParser-Theps_PhraseSlop_Parameter,ps>> |Phrase Slop: specifies the number of positions two terms can be apart in order to match the specified phrase.
-|<<TheDisMaxQueryParser-Theqs_QueryPhraseSlop_Parameter,qs>> |Query Phrase Slop: specifies the number of positions two terms can be apart in order to match the specified phrase. Used specifically with the `qf` parameter.
-|<<TheDisMaxQueryParser-Thetie_TieBreaker_Parameter,tie>> |Tie Breaker: specifies a float value (which should be something much less than 1) to use as tiebreaker in DisMax queries. Default: 0.0
-|<<TheDisMaxQueryParser-Thebq_BoostQuery_Parameter,bq>> |Boost Query: specifies a factor by which a term or phrase should be "boosted" in importance when considering a match.
-|<<TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter,bf>> |Boost Functions: specifies functions to be applied to boosts. (See for details about function queries.)
-|===
+In addition to the common request parameters, highlighting parameters, and simple facet parameters, the DisMax query parser supports the parameters described below. Like the standard query parser, the DisMax query parser allows default parameter values to be specified in `solrconfig.xml`, or overridden by query-time values in the request.
The sections below explain these parameters in detail.
-[[TheDisMaxQueryParser-TheqParameter]]
-=== The q Parameter
+=== q Parameter
The `q` parameter defines the main "query" constituting the essence of the search. The parameter supports raw input strings provided by users with no special escaping. The + and - characters are treated as "mandatory" and "prohibited" modifiers for terms. Text wrapped in balanced quote characters (for example, "San Jose") is treated as a phrase. Any query containing an odd number of quote characters is evaluated as if there were no quote characters at all.
-[IMPORTANT]
-====
-
-The `q` parameter does not support wildcard characters such as *.
+IMPORTANT: The `q` parameter does not support wildcard characters such as *.
-====
-[[TheDisMaxQueryParser-Theq.altParameter]]
-=== The q.alt Parameter
+=== q.alt Parameter
If specified, the `q.alt` parameter defines a query (which by default will be parsed using standard query parsing syntax) when the main q parameter is not specified or is blank. The `q.alt` parameter comes in handy when you need something like a query to match all documents (don't forget `&rows=0` for that one!) in order to get collection-wide faceting counts.
-[[TheDisMaxQueryParser-Theqf_QueryFields_Parameter]]
-=== The qf (Query Fields) Parameter
+=== qf (Query Fields) Parameter
The `qf` parameter introduces a list of fields, each of which is assigned a boost factor to increase or decrease that particular field's importance in the query. For example, the query below:
@@ -85,8 +60,7 @@ The `qf` parameter introduces a list of fields, each of which is assigned a boos
assigns `fieldOne` a boost of 2.3, leaves `fieldTwo` with the default boost (because no boost factor is specified), and `fieldThree` a boost of 0.4. These boost factors make matches in `fieldOne` much more significant than matches in `fieldTwo`, which in turn are much more significant than matches in `fieldThree`.
-[[TheDisMaxQueryParser-Themm_MinimumShouldMatch_Parameter]]
-=== The mm (Minimum Should Match) Parameter
+=== mm (Minimum Should Match) Parameter
When processing queries, Lucene/Solr recognizes three types of clauses: mandatory, prohibited, and "optional" (also known as "should" clauses). By default, all words or phrases specified in the `q` parameter are treated as "optional" clauses unless they are preceded by a "+" or a "-". When dealing with these "optional" clauses, the `mm` parameter makes it possible to say that a certain minimum number of those clauses must match. The DisMax query parser offers great flexibility in how the minimum number can be specified.
@@ -115,27 +89,23 @@ When specifying `mm` values, keep in mind the following:
The default value of `mm` is 100% (meaning that all clauses must match).
-[[TheDisMaxQueryParser-Thepf_PhraseFields_Parameter]]
-=== The pf (Phrase Fields) Parameter
+=== pf (Phrase Fields) Parameter
Once the list of matching documents has been identified using the `fq` and `qf` parameters, the `pf` parameter can be used to "boost" the score of documents in cases where all of the terms in the q parameter appear in close proximity.
The format is the same as that used by the `qf` parameter: a list of fields and "boosts" to associate with each of them when making phrase queries out of the entire q parameter.
-[[TheDisMaxQueryParser-Theps_PhraseSlop_Parameter]]
-=== The ps (Phrase Slop) Parameter
+=== ps (Phrase Slop) Parameter
The `ps` parameter specifies the amount of "phrase slop" to apply to queries specified with the pf parameter. Phrase slop is the number of positions one token needs to be moved in relation to another token in order to match a phrase specified in a query.
-[[TheDisMaxQueryParser-Theqs_QueryPhraseSlop_Parameter]]
-=== The qs (Query Phrase Slop) Parameter
+=== qs (Query Phrase Slop) Parameter
The `qs` parameter specifies the amount of slop permitted on phrase queries explicitly included in the user's query string with the `qf` parameter. As explained above, slop refers to the number of positions one token needs to be moved in relation to another token in order to match a phrase specified in a query.
-[[TheDisMaxQueryParser-Thetie_TieBreaker_Parameter]]
=== The tie (Tie Breaker) Parameter
The `tie` parameter specifies a float value (which should be something much less than 1) to use as tiebreaker in DisMax queries.
@@ -145,8 +115,7 @@ When a term from the user's input is tested against multiple fields, more than o
A value of "0.0" - the default - makes the query a pure "disjunction max query": that is, only the maximum scoring subquery contributes to the final score. A value of "1.0" makes the query a pure "disjunction sum query" where it doesn't matter what the maximum scoring sub query is, because the final score will be the sum of the subquery scores. Typically a low value, such as 0.1, is useful.
-[[TheDisMaxQueryParser-Thebq_BoostQuery_Parameter]]
-=== The bq (Boost Query) Parameter
+=== bq (Boost Query) Parameter
The `bq` parameter specifies an additional, optional, query clause that will be added to the user's main query to influence the score. For example, if you wanted to add a relevancy boost for recent documents:
@@ -159,8 +128,7 @@ bq=date:[NOW/DAY-1YEAR TO NOW/DAY]
You can specify multiple `bq` parameters. If you want your query to be parsed as separate clauses with separate boosts, use multiple `bq` parameters.
-[[TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter]]
-=== The bf (Boost Functions) Parameter
+=== bf (Boost Functions) Parameter
The `bf` parameter specifies functions (with optional boosts) that will be used to construct FunctionQueries which will be added to the user's main query as optional clauses that will influence the score. Any function supported natively by Solr can be used, along with a boost value. For example:
@@ -180,7 +148,7 @@ bf=recip(rord(creationDate),1,1000,1000)
bq={!func}recip(rord(creationDate),1,1000,1000)
----
-[[TheDisMaxQueryParser-ExamplesofQueriesSubmittedtotheDisMaxQueryParser]]
+
== Examples of Queries Submitted to the DisMax Query Parser
All of the sample URLs in this section assume you are running Solr's "techproducts" example:
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8d00e53b/solr/solr-ref-guide/src/the-extended-dismax-query-parser.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/the-extended-dismax-query-parser.adoc b/solr/solr-ref-guide/src/the-extended-dismax-query-parser.adoc
index c4e0fde..3b0bd49 100644
--- a/solr/solr-ref-guide/src/the-extended-dismax-query-parser.adoc
+++ b/solr/solr-ref-guide/src/the-extended-dismax-query-parser.adoc
@@ -1,4 +1,4 @@
-= The Extended DisMax Query Parser
+= The Extended DisMax (eDismax) Query Parser
:page-shortname: the-extended-dismax-query-parser
:page-permalink: the-extended-dismax-query-parser.html
// Licensed to the Apache Software Foundation (ASF) under one
@@ -33,76 +33,52 @@ In addition to supporting all the DisMax query parser parameters, Extended Disma
* supports pure negative nested queries: queries such as `+foo (-foo)` will match all documents.
* lets you specify which fields the end user is allowed to query, and to disallow direct fielded searches.
-[[TheExtendedDisMaxQueryParser-ExtendedDisMaxParameters]]
== Extended DisMax Parameters
-In addition to all the <<the-dismax-query-parser.adoc#TheDisMaxQueryParser-DisMaxParameters,DisMax parameters>>, Extended DisMax includes these query parameters:
+In addition to all the <<the-dismax-query-parser.adoc#dismax-query-parser-parameters,DisMax parameters>>, Extended DisMax includes these query parameters:
-[[TheExtendedDisMaxQueryParser-ThesowParameter]]
-=== The sow Parameter
+`sow`::
+Split on whitespace. If set to `false`, whitespace-separated term sequences will be provided to text analysis in one shot, enabling proper function of analysis filters that operate over term sequences, e.g., multi-word synonyms and shingles. Defaults to `true`, so text analysis is invoked separately for each individual whitespace-separated term.
-Split on whitespace: if set to `false`, whitespace-separated term sequences will be provided to text analysis in one shot, enabling proper function of analysis filters that operate over term sequences, e.g. multi-word synonyms and shingles. Defaults to `true`: text analysis is invoked separately for each individual whitespace-separated term.
-
-[[TheExtendedDisMaxQueryParser-Themm.autoRelaxParameter]]
-=== The mm.autoRelax Parameter
-
-If true, the number of clauses required (<<the-dismax-query-parser.adoc#TheDisMaxQueryParser-Themm_MinimumShouldMatch_Parameter,minimum should match>>) will automatically be relaxed if a clause is removed (by e.g. stopwords filter) from some but not all <<the-dismax-query-parser.adoc#TheDisMaxQueryParser-Theqf_QueryFields_Parameter,`qf`>> fields. Use this parameter as a workaround if you experience that queries return zero hits due to uneven stopword removal between the `qf` fields.
-
-Note that relaxing mm may cause undesired side effects, hurting the precision of the search, depending on the nature of your index content.
-
-[[TheExtendedDisMaxQueryParser-TheboostParameter]]
-=== The boost Parameter
-
-A multivalued list of strings parsed as queries with scores multiplied by the score from the main query for all matching documents. This parameter is shorthand for wrapping the query produced by eDisMax using the `BoostQParserPlugin`
+`mm.autoRelax`::
+If `true`, the number of clauses required (<<the-dismax-query-parser.adoc#mm-minimum-should-match-parameter,minimum should match>>) will automatically be relaxed if a clause is removed (by e.g. stopwords filter) from some but not all <<the-dismax-query-parser.adoc#qf-query-fields-parameter,`qf`>> fields. Use this parameter as a workaround if you experience that queries return zero hits due to uneven stopword removal between the `qf` fields.
++
+Note that relaxing `mm` may cause undesired side effects, such as hurting the precision of the search, depending on the nature of your index content.
-[[TheExtendedDisMaxQueryParser-ThelowercaseOperatorsParameter]]
-=== The lowercaseOperators Parameter
+`boost`::
+A multivalued list of strings parsed as queries with scores multiplied by the score from the main query for all matching documents. This parameter is shorthand for wrapping the query produced by eDisMax using the `BoostQParserPlugin`.
+`lowercaseOperators`::
A Boolean parameter indicating if lowercase "and" and "or" should be treated the same as operators "AND" and "OR".
Defaults to `false`.
-[[TheExtendedDisMaxQueryParser-ThepsParameter]]
-=== The ps Parameter
-
-Default amount of slop on phrase queries built with `pf`, `pf2` and/or `pf3` fields (affects boosting).
+`ps`::
+Phrase Slop. The default amount of slop - distance between terms - on phrase queries built with `pf`, `pf2` and/or `pf3` fields (affects boosting). See also the section <<Using 'Slop'>> below.
-[[TheExtendedDisMaxQueryParser-Thepf2Parameter]]
-=== The pf2 Parameter
+`pf2`::
-A multivalued list of fields with optional weights, based on pairs of word shingles.
-
-[[TheExtendedDisMaxQueryParser-Theps2Parameter]]
-=== The ps2 Parameter
+A multivalued list of fields with optional weights. Similar to `pf`, but based on _pairs_ of word shingles.
+`ps2`::
This is similar to `ps` but overrides the slop factor used for `pf2`. If not specified, `ps` is used.
-[[TheExtendedDisMaxQueryParser-Thepf3Parameter]]
-=== The pf3 Parameter
-
-A multivalued list of fields with optional weights, based on triplets of word shingles. Similar to `pf`, except that instead of building a phrase per field out of all the words in the input, it builds a set of phrases for each field out of each triplet of word shingles.
-
-[[TheExtendedDisMaxQueryParser-Theps3Parameter]]
-=== The ps3 Parameter
+`pf3`::
+A multivalued list of fields with optional weights, based on triplets of word shingles. Similar to `pf`, except that instead of building a phrase per field out of all the words in the input, it builds a set of phrases for each field out of each _triplet_ of word shingles.
+`ps3`::
This is similar to `ps` but overrides the slop factor used for `pf3`. If not specified, `ps` is used.
-[[TheExtendedDisMaxQueryParser-ThestopwordsParameter]]
-=== The stopwords Parameter
-
-A Boolean parameter indicating if the `StopFilterFactory` configured in the query analyzer should be respected when parsing the query: if it is false, then the `StopFilterFactory` in the query analyzer is ignored.
-
-[[TheExtendedDisMaxQueryParser-TheufParameter]]
-=== The uf Parameter
+`stopwords`::
+A Boolean parameter indicating if the `StopFilterFactory` configured in the query analyzer should be respected when parsing the query. If this is set to `false`, then the `StopFilterFactory` in the query analyzer is ignored.
+`uf`::
Specifies which schema fields the end user is allowed to explicitly query. This parameter supports wildcards. The default is to allow all fields, equivalent to `uf=\*`. To allow only title field, use `uf=title`. To allow title and all fields ending with '_s', use `uf=title,*_s`. To allow all fields except title, use `uf=*,-title`. To disallow all fielded searches, use `uf=-*`.
-[[TheExtendedDisMaxQueryParser-Fieldaliasingusingper-fieldqfoverrides]]
-=== Field aliasing using per-field qf overrides
+=== Field Aliasing using Per-Field qf Overrides
Per-field overrides of the `qf` parameter may be specified to provide 1-to-many aliasing from field names specified in the query string, to field names used in the underlying query. By default, no aliasing is used and field names specified in the query string are treated as literal field names in the index.
-[[TheExtendedDisMaxQueryParser-ExamplesofQueriesSubmittedtotheExtendedDisMaxQueryParser]]
-== Examples of Queries Submitted to the Extended DisMax Query Parser
+== Examples of eDismax Queries
All of the sample URLs in this section assume you are running Solr's "```techproducts```" example:
@@ -158,14 +134,12 @@ qf=title text last_name first_name
f.name.qf=last_name first_name
----
-[[TheExtendedDisMaxQueryParser-Usingnegativeboost]]
-== Using negative boost
+== Using Negative Boost
Negative query boosts have been supported at the "Query" object level for a long time (resulting in negative scores for matching documents). Now the QueryParsers have been updated to handle this too.
-[[TheExtendedDisMaxQueryParser-Using_slop_]]
-== Using 'slop'
+== Using 'Slop'
`Dismax` and `Edismax` can run queries against all query fields, and also run a query in the form of a phrase against the phrase fields. (This will work only for boosting documents, not actually for matching.) However, that phrase query can have a 'slop,' which is the distance between the terms of the query while still considering it a phrase match. For example:
@@ -223,8 +197,7 @@ A document that contains "Hans Anderson" will match, but a document that contain
Finally, in addition to the phrase fields (`pf`) parameter, `edismax` also supports the `pf2` and `pf3` parameters, for fields over which to create bigram and trigram phrase queries. The phrase slop for these parameters' queries can be specified using the `ps2` and `ps3` parameters, respectively. If you use `pf2`/`pf3` but `ps2`/`ps3`, then the phrase slop for these parameters' queries will be taken from the `ps` parameter, if any.
-[[TheExtendedDisMaxQueryParser-Usingthe_magicfields__val_and_query_]]
-== Using the "magic fields" \_val_ and \_query_
+== Using the "Magic Fields" \_val_ and \_query_
The Solr Query Parser's use of `\_val_` and `\_query_` differs from the Lucene Query Parser in the following ways:
@@ -257,9 +230,4 @@ createdate:[1976-03-06T23:59:59.999Z TO 1976-03-06T23:59:59.999Z+1YEAR]
createdate:[1976-03-06T23:59:59.999Z/YEAR TO 1976-03-06T23:59:59.999Z]
----
-[IMPORTANT]
-====
-
-`TO` must be uppercase, or Solr will report a 'Range Group' error.
-
-====
+IMPORTANT: `TO` must be uppercase, or Solr will report a 'Range Group' error.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8d00e53b/solr/solr-ref-guide/src/the-query-elevation-component.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/the-query-elevation-component.adoc b/solr/solr-ref-guide/src/the-query-elevation-component.adoc
index dcd3c7e..638aa81 100644
--- a/solr/solr-ref-guide/src/the-query-elevation-component.adoc
+++ b/solr/solr-ref-guide/src/the-query-elevation-component.adoc
@@ -31,7 +31,6 @@ All of the sample configuration and queries used in this section assume you are
bin/solr -e techproducts
----
-[[TheQueryElevationComponent-ConfiguringtheQueryElevationComponent]]
== Configuring the Query Elevation Component
You can configure the Query Elevation Component in the `solrconfig.xml` file. Search components like `QueryElevationComponent` may be added to any request handler; a dedicated request handler is used here for brevity.
@@ -72,7 +71,6 @@ Path to the file that defines query elevation. This file must exist in `<instanc
`forceElevation`::
By default, this component respects the requested `sort` parameter: if the request asks to sort by date, it will order the results by date. If `forceElevation=true` (the default), results will first return the boosted docs, then order by date.
-[[TheQueryElevationComponent-elevate.xml]]
=== elevate.xml
Elevated query results are configured in an external XML file specified in the `config-file` argument. An `elevate.xml` file might look like this:
@@ -95,10 +93,8 @@ Elevated query results are configured in an external XML file specified in the `
In this example, the query "foo bar" would first return documents 1, 2 and 3, then whatever normally appears for the same query. For the query "ipod", it would first return "MA147LL/A", and would make sure that "IW-02" is not in the result set.
-[[TheQueryElevationComponent-UsingtheQueryElevationComponent]]
== Using the Query Elevation Component
-[[TheQueryElevationComponent-TheenableElevationParameter]]
=== The enableElevation Parameter
For debugging it may be useful to see results with and without the elevated docs. To hide results, use `enableElevation=false`:
@@ -107,21 +103,18 @@ For debugging it may be useful to see results with and without the elevated docs
`\http://localhost:8983/solr/techproducts/elevate?q=ipod&df=text&debugQuery=true&enableElevation=false`
-[[TheQueryElevationComponent-TheforceElevationParameter]]
=== The forceElevation Parameter
You can force elevation during runtime by adding `forceElevation=true` to the query URL:
`\http://localhost:8983/solr/techproducts/elevate?q=ipod&df=text&debugQuery=true&enableElevation=true&forceElevation=true`
-[[TheQueryElevationComponent-TheexclusiveParameter]]
=== The exclusive Parameter
You can force Solr to return only the results specified in the elevation file by adding `exclusive=true` to the URL:
`\http://localhost:8983/solr/techproducts/elevate?q=ipod&df=text&debugQuery=true&exclusive=true`
-[[TheQueryElevationComponent-DocumentTransformersandthemarkExcludesParameter]]
=== Document Transformers and the markExcludes Parameter
The `[elevated]` <<transforming-result-documents.adoc#transforming-result-documents,Document Transformer>> can be used to annotate each document with information about whether or not it was elevated:
@@ -132,7 +125,6 @@ Likewise, it can be helpful when troubleshooting to see all matching documents
`\http://localhost:8983/solr/techproducts/elevate?q=ipod&df=text&markExcludes=true&fl=id,[elevated],[excluded]`
-[[TheQueryElevationComponent-TheelevateIdsandexcludeIdsParameters]]
=== The elevateIds and excludeIds Parameters
When the elevation component is in use, the pre-configured list of elevations for a query can be overridden at request time to use the unique keys specified in these request parameters.
@@ -147,7 +139,6 @@ For example, in the request below documents IW-02 and F8V7067-APL-KIT will be el
`\http://localhost:8983/solr/techproducts/elevate?q=ipod&df=text&elevateIds=IW-02,F8V7067-APL-KIT`
-[[TheQueryElevationComponent-ThefqParameter]]
-=== The fq Parameter
+=== The fq Parameter with Elevation
Query elevation respects the standard filter query (`fq`) parameter. That is, if the query contains the `fq` parameter, all results will be within that filter even if `elevate.xml` adds other documents to the result set.