You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Shay Banon <ki...@gmail.com> on 2009/09/05 01:06:19 UTC
CachingTokenFilter#fillCache problem
Hi,
I was going over and playing with Lucene 2.9 and came around something
strange that I think might be a bug. In CachingTokenFilter#fillCache it
seems like captureState is not called on input and instead is called on
CachingTokenFilter. Am I missing something? I got to it since QueryParser
uses the CachingTokenFilter.
Cheers,
Shay
--
View this message in context: http://www.nabble.com/CachingTokenFilter-fillCache-problem-tp25303265p25303265.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: CachingTokenFilter#fillCache problem
Posted by Shay Banon <ki...@gmail.com>.
Ohh, now I see. I missed the fact that everybody across the chain share the
same attributes. Which explain my bug (I forgot to call super at my
tokenizer, thus the attributes were not shared).
Thanks!
Shay
Uwe Schindler wrote:
>
> Hi Shay,
>
>> I was going over and playing with Lucene 2.9 and came around something
>> strange that I think might be a bug. In CachingTokenFilter#fillCache it
>> seems like captureState is not called on input and instead is called on
>> CachingTokenFilter.
>
> This is not a bug and correct behaviour. In general it doesn't matter
> where
> you call captureState(), because all streams and filters in one tokenizer
> chain use exactly the same attribute instances. Calling
> input.incrementToken() moves forward to next token and after that the
> attributes in every filter on the chain contain the same term - just
> because
> they are the same instances.
>
>> Am I missing something? I got to it since QueryParser
>> uses the CachingTokenFilter.
>
> You miss the fact of the shared attributes :-)
>
> Thanks for looking into this and reviewing the code!
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/CachingTokenFilter-fillCache-problem-tp25303265p25307372.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: CachingTokenFilter#fillCache problem
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Shay,
> I was going over and playing with Lucene 2.9 and came around something
> strange that I think might be a bug. In CachingTokenFilter#fillCache it
> seems like captureState is not called on input and instead is called on
> CachingTokenFilter.
This is not a bug and correct behaviour. In general it doesn't matter where
you call captureState(), because all streams and filters in one tokenizer
chain use exactly the same attribute instances. Calling
input.incrementToken() moves forward to next token and after that the
attributes in every filter on the chain contain the same term - just because
they are the same instances.
> Am I missing something? I got to it since QueryParser
> uses the CachingTokenFilter.
You miss the fact of the shared attributes :-)
Thanks for looking into this and reviewing the code!
Uwe
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org