You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2006/02/14 08:31:44 UTC

[Solr Wiki] Trivial Update of "AnalyzersTokenizersTokenFilters" by HossMan

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The following page has been changed by HossMan:
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters

------------------------------------------------------------------------------
  
  Creates `org.apache.lucene.analysis.standard.StandardTokenizer`.
  
- A good general purpose tokenizer that strips many extraneous characters and sets token types to meaningful values.  Token types are only useful for subsequent token filters that are type-aware.  The StandardFilter is currently the only Lucene filter that utilizes token type.
+ A good general purpose tokenizer that strips many extraneous characters and sets token types to meaningful values.  Token types are only useful for subsequent token filters that are type-aware.  The !StandardFilter is currently the only Lucene filter that utilizes token type.
     
  Some token types are number, alphanumeric, email, acronym, URL, etc. &#151;