You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by maxSchlein <m_...@hotmail.com> on 2009/12/24 18:10:05 UTC
help customfilter with incrementToken() and AttributeSource APIs
In the current version of lucene, 3.0 the following methods are no longer
available.
- TokenStream.next()
- TokenStream.next(Token).
- Token.setTermText()
- Token.termText().
The newer versions says to use, incrementToken() and AttributeSource APIs.
But I cannot find much help using these in this way. Any help again is
appreciated. If anyone has a basic example, or can point me to something
useful that would be awesome. Thanx.
Merry Christmas too.
public class CustomAnalyzer extends Analyzer
{
@Override
public TokenStream tokenStream(final String fieldName, final Reader
reader)
{
TokenStream ts = new WhitespaceTokenizer(reader);
ts = new StopFilter(false, ts, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
ts = new LowerCaseFilter(ts);
ts = new CustomFilter(ts);
return ts;
}
}
public class CustomFilter extends TokenFilter
{
protected CustomFilter(TokenStream tokenStream)
{
super(tokenStream);
}
@Override
public Token next(final Token reusableToken) throws IOException
{
Token nextToken = input.next(reusableToken);
if(nextToken != null)
{
nextToken.setTermText(nextToken.termText().replaceAll(":|,|\\(|\\)|“|~|;|&|\\.",""));
}
return nextToken;
}
}
--
View this message in context: http://old.nabble.com/help-customfilter-with-incrementToken%28%29-and-AttributeSource-APIs-tp26915600p26915600.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: help customfilter with incrementToken() and AttributeSource APIs
Posted by Digy <di...@gmail.com>.
The source code for LowerCaseFilter or StopFilter can be a good starting point.
DIGY
-----Original Message-----
From: maxSchlein [mailto:m_schlein@hotmail.com]
Sent: Thursday, December 24, 2009 7:10 PM
To: java-user@lucene.apache.org
Subject: help customfilter with incrementToken() and AttributeSource APIs
In the current version of lucene, 3.0 the following methods are no longer
available.
- TokenStream.next()
- TokenStream.next(Token).
- Token.setTermText()
- Token.termText().
The newer versions says to use, incrementToken() and AttributeSource APIs.
But I cannot find much help using these in this way. Any help again is
appreciated. If anyone has a basic example, or can point me to something
useful that would be awesome. Thanx.
Merry Christmas too.
public class CustomAnalyzer extends Analyzer
{
@Override
public TokenStream tokenStream(final String fieldName, final Reader
reader)
{
TokenStream ts = new WhitespaceTokenizer(reader);
ts = new StopFilter(false, ts, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
ts = new LowerCaseFilter(ts);
ts = new CustomFilter(ts);
return ts;
}
}
public class CustomFilter extends TokenFilter
{
protected CustomFilter(TokenStream tokenStream)
{
super(tokenStream);
}
@Override
public Token next(final Token reusableToken) throws IOException
{
Token nextToken = input.next(reusableToken);
if(nextToken != null)
{
nextToken.setTermText(nextToken.termText().replaceAll(":|,|\\(|\\)|“|~|;|&|\\.",""));
}
return nextToken;
}
}
--
View this message in context: http://old.nabble.com/help-customfilter-with-incrementToken%28%29-and-AttributeSource-APIs-tp26915600p26915600.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org