Override Analyzer.TokenStreamComponents's reader.

[MyCoy Z <my...@gmail.com>]

Hi, Lucene Dev Community:

In our legacy Lucene7 based code, we have logic like:
return new TokenStreamComponents(Tokenizer source, TokenStream result) {
    @Override
    protected void setReader(Final Reader reader) {
        // _config.getCharFilterFactory() , this will return an instance of
org.apache.lucene.analysis.util.CharFilterFactory
        super.setReader(_config.getCharFilterFactory().create(reader));
    }
};

However, in Lucene9.7, Analyzer.TokenStreamComponents.setReader(Reader)
becomes private (actually I don't see this method being called
anywhere in TokenStreamComponents,
not sure why it is still there).
Moreover, Tokenizer.setReader(Reader) is final, and cannot be overridden.

So, could anyone help me to understand what is the best approach to migrate
that piece of legacy code to L9.
An example will also be very helpful.

Thanks

Re: Override Analyzer.TokenStreamComponents's reader.

[Patrick Zhai <zh...@gmail.com>]

Hi MyCoy, according to MIGRATE file
<https://github.com/apache/lucene/blob/main/lucene/MIGRATE.md>

"""
### TokenStreamComponents is now final

Instead of overriding `TokenStreamComponents.setReader()` to customise
analyzer
initialisation, you should now pass a `Consumer<Reader>` instance to the
`TokenStreamComponents` constructor.
"""

Best
Patrick

On Sun, Sep 17, 2023 at 9:26 PM MyCoy Z <my...@gmail.com> wrote:

> Hi, Lucene Dev Community:
>
> In our legacy Lucene7 based code, we have logic like:
> return new TokenStreamComponents(Tokenizer source, TokenStream result) {
>     @Override
>     protected void setReader(Final Reader reader) {
>         // _config.getCharFilterFactory() , this will return an instance
> of org.apache.lucene.analysis.util.CharFilterFactory
>         super.setReader(_config.getCharFilterFactory().create(reader));
>     }
> };
>
> However, in Lucene9.7, Analyzer.TokenStreamComponents.setReader(Reader)
> becomes private (actually I don't see this method being called anywhere in TokenStreamComponents,
> not sure why it is still there).
> Moreover, Tokenizer.setReader(Reader) is final, and cannot be overridden.
>
> So, could anyone help me to understand what is the best approach to
> migrate that piece of legacy code to L9.
> An example will also be very helpful.
>
> Thanks
>
>