You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Tom Klonikowski (Created) (JIRA)" <ji...@apache.org> on 2011/11/21 11:52:51 UTC
[jira] [Created] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
----------------------------------------------------------------------------------
Key: SOLR-2909
URL: https://issues.apache.org/jira/browse/SOLR-2909
Project: Solr
Issue Type: Improvement
Components: Schema and Analysis
Affects Versions: 3.4, 4.0
Reporter: Tom Klonikowski
Priority: Minor
The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
{noformat}
private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
tokFactory.init(args);
if (tokFactory instanceof ResourceLoaderAware) {
((ResourceLoaderAware) tokFactory).inform(loader);
}
return tokFactory;
}
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211802#comment-13211802 ]
Robert Muir commented on SOLR-2909:
-----------------------------------
+1, good catch.
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Koji Sekiguchi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211701#comment-13211701 ]
Koji Sekiguchi commented on SOLR-2909:
--------------------------------------
I've talked with Ohtani-san for this issue. He hit this problem and got NPE when using another JapaneseTokenizer (lucene-gosen) with SynonymFilterFactory.
As I agree with him about that the patch looks nice provided in the Description section above, I'd like to commit if nobody objects.
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Koji Sekiguchi (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi reassigned SOLR-2909:
------------------------------------
Assignee: Koji Sekiguchi
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Koji Sekiguchi (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi resolved SOLR-2909.
----------------------------------
Resolution: Fixed
Fix Version/s: 4.0
3.6
trunk: Committed revision 1291541.
branch_3x: Committed revision 1291543.
Thanks, everyone!
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-2909.patch
>
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Koji Sekiguchi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-2909:
---------------------------------
Attachment: SOLR-2909.patch
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
> Attachments: SOLR-2909.patch
>
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Christian Moen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211711#comment-13211711 ]
Christian Moen commented on SOLR-2909:
--------------------------------------
Ohtani-san, good catch. Sekiguchi-san, this sounds like a good idea to me.
My understanding is that the problem also occurs when using {{SynonymFilterFactory}} and that the above will address this case as well since the actual synonym handling is being done by {{SlowSynonymFilterFactory}} or {{FSTSynonymFilterFactory}} through delegation (based on version). Is this correct? Thanks.
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Jun Ohtani (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211698#comment-13211698 ]
Jun Ohtani commented on SOLR-2909:
----------------------------------
A problem seems to happen when using TokenizerFactory of Kuromoji.
KuromojiTokenizerFactory load userDictionaryPath and mode in inform().
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Christian Moen (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211736#comment-13211736 ]
Christian Moen commented on SOLR-2909:
--------------------------------------
Thanks.
Sekiguchi-san, I'm happy to test and review a patch if you'd like that.
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2909) Add support for ResourceLoaderAware
tokenizerFactories in synonym filter factories
Posted by "Jun Ohtani (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211721#comment-13211721 ]
Jun Ohtani commented on SOLR-2909:
----------------------------------
Yes.
Probably the word of the user's dictionary is not used in SynonymFilter.
> Add support for ResourceLoaderAware tokenizerFactories in synonym filter factories
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2909
> URL: https://issues.apache.org/jira/browse/SOLR-2909
> Project: Solr
> Issue Type: Improvement
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Reporter: Tom Klonikowski
> Assignee: Koji Sekiguchi
> Priority: Minor
>
> The optional custom tokenizerFactory in SlowSynonymFilterFactory and FSTSynonymFilterFactory might require the ResourceLoader information. Thus inform(ResourceLoader) should be called if the specified tokenizerFactory is an instance of ResourceLoaderAware.
> {noformat}
> private static TokenizerFactory loadTokenizerFactory(ResourceLoader loader, String cname, Map<String, String> args) {
> TokenizerFactory tokFactory = (TokenizerFactory) loader.newInstance(cname);
> tokFactory.init(args);
> if (tokFactory instanceof ResourceLoaderAware) {
> ((ResourceLoaderAware) tokFactory).inform(loader);
> }
> return tokFactory;
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org