Why next(Token) in CharTokenizer is final?

[OBender <os...@hotmail.com>]

Hi All,

I think this is a question to Lucene dev team.
Why the next(Token) method of CharTokenizer was made final?
It is quite inconvenient and I don't see the reason why it is so.

Thanks.



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

RE: Why next(Token) in CharTokenizer is final?

[Uwe Schindler <uw...@thetaphi.de>]

In general all TokenStream classes from Lucene core are/should be final,
because should simply put an additional TokenFilter into the chain to modify
the tokenization.

A few of them are not final (CharTokenizer is abstract), but here the next()
methods are final because of the above explained reason, because you should
only be able to customize the behaviour (which chars are whitespace and how
they are "normalized"), not the implementation of the TokenStream (which
could break a lot of things, see below).

If you want to plug in additional custom filtering into the TokenStream
chain, create a new TokenFilter and plug it ontop of CharTokenizer
subclasses, do not modify the core classes!

And this is now very important for the transition to the new TokenStream
API, see https://issues.apache.org/jira/browse/LUCENE-1693

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: OBender [mailto:osya_bender@hotmail.com]
> Sent: Friday, July 17, 2009 10:47 PM
> To: java-user@lucene.apache.org
> Subject: Why next(Token) in CharTokenizer is final?
> 
> Hi All,
> 
> I think this is a question to Lucene dev team.
> Why the next(Token) method of CharTokenizer was made final?
> It is quite inconvenient and I don't see the reason why it is so.
> 
> Thanks.
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org