[GitHub] [lucenenet] NightOwl888 commented on issue #271: TokenStream.IncrementToken() is called after Dispose() is called

[GitBox <gi...@apache.org>]

NightOwl888 commented on issue #271:
URL: https://github.com/apache/lucenenet/issues/271#issuecomment-657130656


   I looked into this a little, and it appears that the bug that was fixed in the test framework didn't solve this. When adding a `Dispose(bool)` override to the `ICUCollationKeyFilter`, `Dispose(true)` is still being called followed by additional calls to `IncrementToken()`.
   
   This is occuring in `DocFieldProcessor` which [does a tight loop](https://github.com/apache/lucenenet/blob/081edeed35b190c1d535dcfdfeb91143f0ef818f/src/Lucene.Net/Index/DocFieldProcessor.cs#L280-L284) around some fields. It is possible that the consumer will dispose the `TokenFilter` in one loop and then another loop will retrieve the same instance of that `TokenFilter` by calling `Field.GetTokenStream(Analyzer)` and call `IncrementToken()` on it after it is disposed.
   
   At this point I haven't confirmed this is how it works in Java so it is still unclear if this is expected behavior or a bug. If the former, we should consider the possibility of making `TokenStream` "close" and do that separately than "dispose" (see #265).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org