[jira] Commented: (LUCENE-1377) Add HTMLStripReader and WordDelimiterFilter from SOLR

["Robert Muir (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715595#action_12715595 ] 

Robert Muir commented on LUCENE-1377:
-------------------------------------

the WordDelimiterFilter in SOLR trunk as it stands today would be a significant benefit to lucene.

Also, I think there's a very valuable use for an analyzer something like the following:
WhitespaceTokenizer
WordDelimiterFilter (default settings)
LowerCaseFilter

This simple configuration would provide some much needed functionality to lucene, specifically the ability to index things like Hindi text. Its not perfect and will add some additional nonsense terms for some languages, but in the short term, is much more friendly to a variety of languages where no viable option in lucene exists at all right now.


> Add HTMLStripReader and WordDelimiterFilter from SOLR
> -----------------------------------------------------
>
>                 Key: LUCENE-1377
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1377
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>    Affects Versions: 2.3.2
>            Reporter: Jason Rutherglen
>            Priority: Minor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> SOLR has two classes HTMLStripReader and WordDelimiterFilter which are very useful for a wide variety of use cases.  It would be good to place them into core Lucene.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org