You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2013/07/08 17:23:48 UTC
[jira] [Updated] (LUCENE-5097) Add utility method to Analyzer:
public final TokenStream tokenStream(String fieldName,String text)
[ https://issues.apache.org/jira/browse/LUCENE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-5097:
----------------------------------
Attachment: LUCENE-5097.patch
Quick patch for demonstration purposes:
- Moved the ReusableStringReader out of Field.java to the analysis package (pkg-private - could also be a inner class in Analyzer; I did this because I wanted a separate test)
- added a second tokenStream method that lazy inits the reusable reader and stores it in a hidden transient field of TokenStreamComponents
This is all still a little bit hackish, but shows my idea. By this you can reuse the StringReader (without synchronization cost) and we dont need extra code in Field.java handling the field reuse.
> Add utility method to Analyzer: public final TokenStream tokenStream(String fieldName,String text)
> --------------------------------------------------------------------------------------------------
>
> Key: LUCENE-5097
> URL: https://issues.apache.org/jira/browse/LUCENE-5097
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
> Affects Versions: 4.3.1
> Reporter: Uwe Schindler
> Attachments: LUCENE-5097.patch
>
>
> It might be a good idea to remove tons of useless code from tests:
> Most people use TokenStreams and Analyzers by only passing a String, wrapped by a StringReader. It would make life easier, if Analyzer would have an additional public (and final!!!) method that simply does the wrapping with StringReader by itsself. It might maybe not even needed to throw IOException (not sure)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org