You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2013/09/12 13:44:00 UTC

[Solr Wiki] Update of "ClusteringComponent" by DawidWeiss

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "ClusteringComponent" page has been changed by DawidWeiss:
https://wiki.apache.org/solr/ClusteringComponent?action=diff&rev1=61&rev2=62

  = Clustering Component =
  
- This [[SearchComponent|search component]] can cluster both [[#Search_Results_Clustering|search results]] and [[#Document_Clustering|documents]]. In case you're wondering what clustering is good for, think of it as a quick way to summarize a whole bunch of results/documents, or as a way to group together semantically related results/documents. See http://en.wikipedia.org/wiki/Data_clustering for more background, and [[http://search.carrot2.org/stable/search?query=apache|Carrot2 demo]] for a live example.
+ This [[SearchComponent|search component]] can cluster both [[#Search_Results_Clustering|search results]] and [[#Document_Clustering|documents]]. In case you're wondering what clustering is good for, think of it as a quick way to summarize a whole bunch of results/documents, or as a way to group together semantically related results/documents. See http://en.wikipedia.org/wiki/Data_clustering for more background, and [[http://search.carrot2.org/stable/search?query=apache|Carrot2 demo]] for a live example. More information about Solr-Carrot^2^ integration strategies, including per-Solr-version examples are available at [[http://carrot2.github.io/solr-integration-strategies/|http://carrot2.github.io/solr-integration-strategies/]]
  
  Jump to: [[#Quick_Start|Quick Start]] | [[#Configuration|Configuration]] | [[#Parameters|Parameters]]
  
@@ -163, +163 @@

  
   The request handler configuration needs to set a number of common parameters, such as the type of clustering ([[#clustering.results|clustering.results]]), and a number of engine-specific parameters, e.g. fields to cluster on ([[#carrot.title|carrot.title]], [[#carrot.snippet|carrot.snippet]]).
  
+ <!> [[Solr4.5]] Starting with Solr 4.5 there is no need to declare a "default" engine. The first declared engine becomes the default one and it can be overridden with [[#clustering.engine|clustering.engine]] request parameter.
+ 
  = Parameters =
  
  This section lists common parameters that apply to both search results and document clustering. Please also see the [[#Parameters|search results clustering parameters]].
@@ -173, +175 @@

  
  == clustering.engine ==
  
- The clustering engine to use. If not specified, the engine named `default` will be used.
+ The clustering engine to use. If not specified, the engine named `default` will be used (or, starting with <!> [[Solr4.5]], the first declared engine).
  
  == clustering.results ==
  
@@ -215, +217 @@

  
  === carrot.title ===
  
- The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the search results's title. All fields must be stored fields.
+ The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the hit document's title. It must be a stored field (or fields).
  
- Carrot^2^ will give more weight to the content of this field compared to [[#carrot.snippet|carrot.snippet]]. For best results, the field should contain concise, noise-free content, such as the document's title.
+ Carrot^2^ will give more weight to the content of this field compared to [[#carrot.snippet|carrot.snippet]]. For best results, the field should contain concise, noise-free content.
  
  If your schema does not distinguish the document's title and content, you can provide your content in [[#carrot.title|carrot.title]] and leave [[#carrot.snippet|carrot.snippet]] empty.
  
  
  === carrot.snippet ===
  
- The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the search results's content. All fields must be stored fields.
+ The Solr field ( <!> [[Solr3.6]]: comma- or space-separated list of fields) that the clustering engine should treat as the hit document's content. It must be a stored field (or fields).
  
- For best results, the snippet field should contain a summary of the document, e.g. an abstract or the first content paragraph. Very long snippet fields will significantly increase the clustering time, unless [[#carrot.produceSummary|carrot.produceSummary]] is enabled.
+ For best results, the snippet should contain a summary of the document, e.g. an abstract or the first content paragraph. Very long snippet fields will significantly increase the clustering time, unless [[#carrot.produceSummary|carrot.produceSummary]] is enabled.
  
  
  === carrot.url ===
  
- The Solr field that the clustering engine should treat as the search result's URL. Must be a stored field, optional.
+ The Solr field that the clustering engine should treat as the hit document's target URL. Must be a stored field. This mapping is optional.
  
  The URL field is currently not used by the Carrot^2^ algorithms.
  
@@ -290, +292 @@

  
  <!> [[Solr3.2]]
  <!> [[Solr4.0]]
+ <!> [[Solr4.5]] (deprecated, use [[#carrot.resourcesDir|carrot.resourcesDir]]).
  
  Specifies the directory from which Carrot^2^ should load its lexical resources, such as stop words and stop labels files. For more information on the syntax of these files, see the [[http://download.carrot2.org/head/manual/#chapter.lexical-resources|overview of lexical resources in Carrot2 manual]].
  
- The lexical resources directory can be either absolute ( <!> [[Solr3.4]]) or relative to `${solr.home}/conf`. The default is: `clustering/carrot2`, relative to `${solr.home}/conf`.
+ The lexical resources directory can be either absolute ( <!> [[Solr3.4]]) or relative to `${solr.home}/conf`. The default is: `clustering/carrot`, relative to `${solr.home}/conf`.
  
  If a specific Carrot^2^ resource (e.g. `stopwords.en`) is present in the specified dir, it will completely override the corresponding default one that ships with Carrot^2^.
  
  '''Note:''' Carrot^2^ caches its lexical resources by default. The cache can be flushed either by restarting Solr or by appending the `&reload-resources=true` parameter to the request URL. Please note that resource reloading significantly increases the clustering time, so it should not be used when running regular production queries.
  
+ === carrot.resourcesDir ===
+ 
+ <!> [[Solr4.5]]
+ 
+ Specifies a directory with optional resources overriding Carrot^2^ defaults, much like [[#carrot.lexicalResourcesDir|carrot.lexicalResourcesDir]]. In addition to that, this folder may contain per-engine attribute XML files exported from the Carrot^2^ workbench and configuring each algorithm. An attribute file for an engine `XYZ` is expected to be named, by convention, `XYZ-attributes.xml`. See the default Solr example work an example configuration of STC, Lingo and bisecting k-means.
  
  === Carrot2-specific parameters ===
  
- Parameters of a specific clustering algorithm, e.g. `LingoClusteringAlgorithm.desiredClusterCountBase` can also be specified. A complete list of clustering algorithm is available in Carrot2 documentation:
+ Parameters of a specific clustering algorithm, e.g. `LingoClusteringAlgorithm.desiredClusterCountBase` can also be specified. A complete list of attributes for each clustering algorithm is available in Carrot2 documentation:
  
   * [[http://download.carrot2.org/head/manual/#section.component.lingo|Lingo clustering algorithm parameters]]
   * [[http://download.carrot2.org/head/manual/#section.component.stc|STC clustering algorithm parameters]]
@@ -311, +319 @@

  You can specify clustering algorithm parameters both in `solrconfig.xml` and at request time, e.g.:
  
  http://localhost:8983/solr/clustering?q=*:*&rows=10&LingoClusteringAlgorithm.desiredClusterCountBase=20
+ 
+ <!> [[Solr4.5]] Starting with Solr 4.5, the preferred way of configuring clustering algorithms is to export an XML file with attributes from Carrot^2^ Workbench and place such a file in [[#carrot.resourcesDir|carrot.resourcesDir]], named `enginename-attributes.xml`.
     
  
  == Performance impact ==