You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2010/07/09 17:45:06 UTC

[Solr Wiki] Update of "LanguageAnalysis" by RobertMuir

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "LanguageAnalysis" page has been changed by RobertMuir.
The comment on this change is: add mention for hyphenation-based decompounding.
http://wiki.apache.org/solr/LanguageAnalysis?action=diff&rev1=2&rev2=3

--------------------------------------------------

  
  Solr provides dictionary-based decompounding support via solr.DictionaryCompoundWordTokenFilterFactory. This factory allows you to provide a dictionary, along with some settings (min/max subword size, etc), to break compound words into pieces.
  
+ <!> [[Solr3.1]]
+ 
+ Additionally, you can use solr.HyphenationCompoundWordTokenFilterFactory. This factory uses a hyphenation grammar in combination with an optional dictionary to break compound words into pieces. Hyphenation grammars for a few languages can be found at the [[http://offo.sourceforge.net/hyphenation/|FOP XML Hyphenation Patterns]] site.
+ 
  One alternative is to use n-gram tokenization so that the search is less sensitive to compound words.
  
- TODO: Add support for Lucene's hyphenation grammar-based decompounding and document it here.
-