You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Michael Sokolov (Jira)" <ji...@apache.org> on 2022/05/04 21:53:00 UTC

[jira] [Comment Edited] (LUCENE-10558) Expose IOSupplier constructors in Kuromoji (and Nori?)

    [ https://issues.apache.org/jira/browse/LUCENE-10558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531960#comment-17531960 ] 

Michael Sokolov edited comment on LUCENE-10558 at 5/4/22 9:52 PM:
------------------------------------------------------------------

Hi [~uschindler] – we have code like this:

 

 

{{ connectionCosts = new ConnectionCosts(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/ConnectionCosts");}}
{{ systemDictionary = new TokenInfoDictionary(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/TokenInfoDictionary");}}
{{ unknownDictionary = new UnknownDictionary(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/UnknownDictionary");}}

{{ japaneseTokenizer = new JapaneseTokenizer(attributeFactory, systemDictionary,}}
{{                unknownDictionary, connectionCosts, userDict, getDiscardPunctuationSetting(context),}}
{{                false, mode);}}
  

 

It stops using those custom resources in 9.1


was (Author: sokolov):
Hi [~uschindler] – we have code like this:

 

{{

            connectionCosts = new ConnectionCosts(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/ConnectionCosts");
            systemDictionary = new TokenInfoDictionary(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/TokenInfoDictionary");
            unknownDictionary = new UnknownDictionary(ResourceScheme.CLASSPATH, DICTIONARY_PATH + "/UnknownDictionary");

            japaneseTokenizer = new JapaneseTokenizer(attributeFactory, systemDictionary,
                unknownDictionary, connectionCosts, userDict, getDiscardPunctuationSetting(context),
                false, mode);
        }}

 

It stops using those custom resources in 9.1

> Expose IOSupplier<InputStream> constructors in Kuromoji (and Nori?)
> -------------------------------------------------------------------
>
>                 Key: LUCENE-10558
>                 URL: https://issues.apache.org/jira/browse/LUCENE-10558
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael Sokolov
>            Priority: Major
>
> When we refactored the constructors for  these resource objects used by the kuromoji JapaneseTokenizer,  we (inadvertently, I expect) changed the behavior for consumers that were supplying these resources on the classpath. In that case, we silently replaced the custom resources with the Lucene built-in ones.  I think we cannot support the old API because of Java Module system restrictions, but we didn't provide any usable replacement or notice either.
>  
> This issue is for exposing the new (private) constructors that accept streams, and adding a notice to Migration.md to point users at them, since they can be used with resources streams loaded from the classpath by the caller.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org