You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-commits@lucene.apache.org by Apache Wiki <wi...@apache.org> on 2011/04/18 06:31:51 UTC

[Solr Wiki] Update of "AnalyzersTokenizersTokenFilters" by MikeThomas

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "AnalyzersTokenizersTokenFilters" page has been changed by MikeThomas.
The comment on this change is: Removing discussion of HTMLStrip*Tokenizers, since they have been deleted in favor of using HTMLStripCharFilterFactory followed by a tokenizer of choice.
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters?action=diff&rev1=112&rev2=113

--------------------------------------------------

  ||'''arg''' ||'''default value''' ||'''note''' ||
  ||maxTokenLength ||255 || <!> [[Solr3.1]] -- [[https://issues.apache.org/jira/browse/SOLR-2188|SOLR-2188]]<<BR>>Tokens longer than `maxTokenLength` are silently ignored. ||
  
- 
- <<Anchor(HTMLStripWhitespaceTokenizer)>>
- 
- === solr.HTMLStripWhitespaceTokenizerFactory ===
- Strips HTML from the input stream and passes the result to a !WhitespaceTokenizer.
- 
- See {{{solr.HTMLStripCharFilterFactory}}} for details on HTML stripping.
- 
- === solr.HTMLStripStandardTokenizerFactory ===
- Strips HTML from the input stream and passes the result to a !StandardTokenizer.
- 
- See {{{solr.HTMLStripCharFilterFactory}}} for details on HTML stripping.
  
  === solr.PatternTokenizerFactory ===
  Breaks text at the specified regular expression pattern.