You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2013/09/12 13:44:00 UTC
[Solr Wiki] Update of "ClusteringComponent" by DawidWeiss
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.
The "ClusteringComponent" page has been changed by DawidWeiss:
https://wiki.apache.org/solr/ClusteringComponent?action=diff&rev1=61&rev2=62
= Clustering Component =
- This [[SearchComponent|search component]] can cluster both [[#Search_Results_Clustering|search results]] and [[#Document_Clustering|documents]]. In case you're wondering what clustering is good for, think of it as a quick way to summarize a whole bunch of results/documents, or as a way to group together semantically related results/documents. See http://en.wikipedia.org/wiki/Data_clustering for more background, and [[http://search.carrot2.org/stable/search?query=apache|Carrot2 demo]] for a live example.
+ This [[SearchComponent|search component]] can cluster both [[#Search_Results_Clustering|search results]] and [[#Document_Clustering|documents]]. In case you're wondering what clustering is good for, think of it as a quick way to summarize a whole bunch of results/documents, or as a way to group together semantically related results/documents. See http://en.wikipedia.org/wiki/Data_clustering for more background, and [[http://search.carrot2.org/stable/search?query=apache|Carrot2 demo]] for a live example. More information about Solr-Carrot^2^ integration strategies, including per-Solr-version examples are available at [[http://carrot2.github.io/solr-integration-strategies/|http://carrot2.github.io/solr-integration-strategies/]]
Jump to: [[#Quick_Start|Quick Start]] | [[#Configuration|Configuration]] | [[#Parameters|Parameters]]
@@ -163, +163 @@
The request handler configuration needs to set a number of common parameters, such as the type of clustering ([[#clustering.results|clustering.results]]), and a number of engine-specific parameters, e.g. fields to cluster on ([[#carrot.title|carrot.title]], [[#carrot.snippet|carrot.snippet]]).
+ <!> [[Solr4.5]] Starting with Solr 4.5 there is no need to declare a "default" engine. The first declared engine becomes the default one and it can be overridden with [[#clustering.engine|clustering.engine]] request parameter.
+
= Parameters =
This section lists common parameters that apply to both search results and document clustering. Please also see the [[#Parameters|search results clustering parameters]].
@@ -173, +175 @@
== clustering.engine ==
- The clustering engine to use. If not specified, the engine named `default` will be used.
+ The clustering engine to use. If not specified, the engine named `default` will be used (or, starting with <!> [[Solr4.5]], the first declared engine).
== clustering.results ==
@@ -215, +217 @@
=== carrot.title ===
- The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the search results's title. All fields must be stored fields.
+ The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the hit document's title. It must be a stored field (or fields).
- Carrot^2^ will give more weight to the content of this field compared to [[#carrot.snippet|carrot.snippet]]. For best results, the field should contain concise, noise-free content, such as the document's title.
+ Carrot^2^ will give more weight to the content of this field compared to [[#carrot.snippet|carrot.snippet]]. For best results, the field should contain concise, noise-free content.
If your schema does not distinguish the document's title and content, you can provide your content in [[#carrot.title|carrot.title]] and leave [[#carrot.snippet|carrot.snippet]] empty.
=== carrot.snippet ===
- The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the search results's content. All fields must be stored fields.
+ The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the hit document's content. It must be a stored field (or fields).
- For best results, the snippet field should contain a summary of the document, e.g. an abstract or the first content paragraph. Very long snippet fields will significantly increase the clustering time, unless [[#carrot.produceSummary|carrot.produceSummary]] is enabled.
+ For best results, the snippet should contain a summary of the document, e.g. an abstract or the first content paragraph. Very long snippet fields will significantly increase the clustering time, unless [[#carrot.produceSummary|carrot.produceSummary]] is enabled.
=== carrot.url ===
- The Solr field that the clustering engine should treat as the search result's URL. Must be a stored field, optional.
+ The Solr field that the clustering engine should treat as the hit document's target URL. Must be a stored field. This mapping is optional.
The URL field is currently not used by the Carrot^2^ algorithms.
@@ -290, +292 @@
<!> [[Solr3.2]]
<!> [[Solr4.0]]
+ <!> [[Solr4.5]] (deprecated, use [[#carrot.resourcesDir|carrot.resourcesDir]]).
Specifies the directory from which Carrot^2^ should load its lexical resources, such as stop words and stop labels files. For more information on the syntax of these files, see the [[http://download.carrot2.org/head/manual/#chapter.lexical-resources|overview of lexical resources in Carrot2 manual]].
- The lexical resources directory can be either absolute ( <!> [[Solr3.4]]) or relative to `${solr.home}/conf`. The default is: `clustering/carrot2`, relative to `${solr.home}/conf`.
+ The lexical resources directory can be either absolute ( <!> [[Solr3.4]]) or relative to `${solr.home}/conf`. The default is: `clustering/carrot`, relative to `${solr.home}/conf`.
If a specific Carrot^2^ resource (e.g. `stopwords.en`) is present in the specified dir, it will completely override the corresponding default one that ships with Carrot^2^.
'''Note:''' Carrot^2^ caches its lexical resources by default. The cache can be flushed either by restarting Solr or by appending the `&reload-resources=true` parameter to the request URL. Please note that resource reloading significantly increases the clustering time, so it should not be used when running regular production queries.
+ === carrot.resourcesDir ===
+
+ <!> [[Solr4.5]]
+
+ Specifies a directory with optional resources overriding Carrot^2^ defaults, much like [[#carrot.lexicalResourcesDir|carrot.lexicalResourcesDir]]. In addition to that, this folder may contain per-engine attribute XML files exported from the Carrot^2^ workbench and configuring each algorithm. An attribute file for an engine `XYZ` is expected to be named, by convention, `XYZ-attributes.xml`. See the default Solr example work an example configuration of STC, Lingo and bisecting k-means.
=== Carrot2-specific parameters ===
- Parameters of a specific clustering algorithm, e.g. `LingoClusteringAlgorithm.desiredClusterCountBase` can also be specified. A complete list of clustering algorithm is available in Carrot2 documentation:
+ Parameters of a specific clustering algorithm, e.g. `LingoClusteringAlgorithm.desiredClusterCountBase` can also be specified. A complete list of attributes for each clustering algorithm is available in Carrot2 documentation:
* [[http://download.carrot2.org/head/manual/#section.component.lingo|Lingo clustering algorithm parameters]]
* [[http://download.carrot2.org/head/manual/#section.component.stc|STC clustering algorithm parameters]]
@@ -311, +319 @@
You can specify clustering algorithm parameters both in `solrconfig.xml` and at request time, e.g.:
http://localhost:8983/solr/clustering?q=*:*&rows=10&LingoClusteringAlgorithm.desiredClusterCountBase=20
+
+ <!> [[Solr4.5]] Starting with Solr 4.5, the preferred way of configuring clustering algorithms is to export an XML file with attributes from Carrot^2^ Workbench and place such a file in [[#carrot.resourcesDir|carrot.resourcesDir]], named `enginename-attributes.xml`.
== Performance impact ==