You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org> on 2009/07/23 08:19:14 UTC
[jira] Created: (SOLR-1307) Provide a way to reload plugins
Provide a way to reload plugins
-------------------------------
Key: SOLR-1307
URL: https://issues.apache.org/jira/browse/SOLR-1307
Project: Solr
Issue Type: New Feature
Components: search, update
Reporter: Shalin Shekhar Mangar
Fix For: 1.5
Currently, Solr plugins have no way to reload themselves. Therefore, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
We should provide a way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1307) Provide a standard way to
reload plugins
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736496#action_12736496 ]
Noble Paul edited comment on SOLR-1307 at 7/29/09 12:59 AM:
------------------------------------------------------------
The possible events as I see are config file changes , commit /optimize. commit/optimize are already handled . so the main requirement is to make it possible for Solr to 'watch' a few files .
say
{code:java}
core.registerConfFileListener(listener, List<String> files);
{code}
this would need a new interface
{code:java}
public interface ConfFileListener {
public void confFileChanged(List<String> changedFiles);
}
{code}
was (Author: noble.paul):
The possible events as I see are config file changes , commit /optimize. commit/optimize are already handled . so the main requirement is to make it possible for Solr to 'watch' a few files .
say
{code:java}
core.registerConfFileListener(listener, List<String> files);
{code}
this would need a new interface
{code:java}
public interface ConfFileListener {
public void confFileChanged(List<String> files);
}
{code}
> Provide a standard way to reload plugins
> ----------------------------------------
>
> Key: SOLR-1307
> URL: https://issues.apache.org/jira/browse/SOLR-1307
> Project: Solr
> Issue Type: New Feature
> Components: search, update
> Reporter: Shalin Shekhar Mangar
> Fix For: 1.5
>
>
> Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
> We should provide a standard way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1307) Provide a standard way to reload
plugins
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736490#action_12736490 ]
Shalin Shekhar Mangar commented on SOLR-1307:
---------------------------------------------
bq. this could be very hairy depending on what you mean by "reload"
Yeah, I kept it vague on purpose to have this discussion first.
{quote}
when we talk about reloading cores, what happens is we instantiate a new core, then replace refrences to the old core with refrences to the new core.
attempting to do something similar with each and every type of plugin seems like it could get incredibly tedious to deal with the synchronization issues.
{quote}
I had thought about a new instance swapping out the old instance. I actually thought that it may be the easiest approach. But then I realized that it may not be a good approach for resource intensive plugins e.g. a SpellCheckComponent managing several dictionaries. One wouldn't want to create two instances each with their own dictionaries. An easier option would be to just have an interface with a reload method which plugins can implement and let them do whatever they need to do. The onus of thread-safety will be on the plugins themselves.
{quote}
This is actually something that's already possible now: any plugin can spin up a TimerTask to reload things in a background thread, SolrCoreAware plugins can programaticly register newSearcher listeners to make callbacks, etc... We probably just want to provide helper code to make this easier for people, and add the functionality to some of the obvious choices in the plugins that ship with Solr along with options to enable it (you might want a query analyzer instance of SynonymFilterFactory to md5/reload the synonyms file on every commit, but an index analyzer instance of SYnonymFilterFactory probably shouldn't)
{quote}
Yes, it is possible now. Just that everyone does it their own way and some don't at all. DIH has an HTTP API for reload. SpellCheckComponent has reload/build as HTTP API and buildOnCommit/buildOnOptimize configuration. QueryElevationComponent, SynonymFilter have nothing.
All I want is to have some consistency in implementing and invoking such operations. Look at SpellCheckComponent's SpellCheckListener for the kind of hacks we've had to do to implement these features.
Perhaps all we need is a better event listener API? I wouldn't want to have complex configuration to register components with certain kinds of events. As we have limited number of events and limited number of components, perhaps we can pass all events into a plugin and let it decide what it wants to do on that event?
> Provide a standard way to reload plugins
> ----------------------------------------
>
> Key: SOLR-1307
> URL: https://issues.apache.org/jira/browse/SOLR-1307
> Project: Solr
> Issue Type: New Feature
> Components: search, update
> Reporter: Shalin Shekhar Mangar
> Fix For: 1.5
>
>
> Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
> We should provide a standard way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1307) Provide a standard way to reload
plugins
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736234#action_12736234 ]
Hoss Man commented on SOLR-1307:
--------------------------------
this could be very hairy depending on what you mean by "reload"
when we talk about reloading cores, what happens is we instantiate a new core, then replace refrences to the old core with refrences to the new core.
attempting to do something similar with each and every type of plugin seems like it could get incredibly tedious to deal with the synchronization issues.
what seems like it might make the most sense for the common case is to have helper code available to make it easier for plugins to do their own internal reloading (ie: instead of constructing a new instance of SynonymFIlterFactory, we make it easy for an instance of SynomymFilterFactory to reload it's own synonym text file). going this route means that only the classes that *want* to be reloadable need to worry about the synchronization costs associated with doing so (ie: StopWorkFilterFactory can put synchronization on some getStopWordSet() method it uses when constructing each StopFilter instance)
This is actually something that's already possible now: any plugin can spin up a TimerTask to reload things in a background thread, SolrCoreAware plugins can programaticly register newSearcher listeners to make callbacks, etc... We probably just want to provide helper code to make this easier for people, and add the functionality to some of the obvious choices in the plugins that ship with Solr along with options to enable it (you might want a query analyzer instance of SynonymFilterFactory to md5/reload the synonyms file on every commit, but an index analyzer instance of SYnonymFilterFactory probably shouldn't)
> Provide a standard way to reload plugins
> ----------------------------------------
>
> Key: SOLR-1307
> URL: https://issues.apache.org/jira/browse/SOLR-1307
> Project: Solr
> Issue Type: New Feature
> Components: search, update
> Reporter: Shalin Shekhar Mangar
> Fix For: 1.5
>
>
> Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
> We should provide a standard way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1307) Provide a standard way to reload
plugins
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1307:
----------------------------------------
Description:
Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
We should provide a standard way for plugins to reload themselves on events relevant to them.
was:
Currently, Solr plugins have no way to reload themselves. Therefore, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
We should provide a way for plugins to reload themselves on events relevant to them.
Summary: Provide a standard way to reload plugins (was: Provide a way to reload plugins)
> Provide a standard way to reload plugins
> ----------------------------------------
>
> Key: SOLR-1307
> URL: https://issues.apache.org/jira/browse/SOLR-1307
> Project: Solr
> Issue Type: New Feature
> Components: search, update
> Reporter: Shalin Shekhar Mangar
> Fix For: 1.5
>
>
> Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
> We should provide a standard way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1307) Provide a standard way to reload
plugins
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736496#action_12736496 ]
Noble Paul commented on SOLR-1307:
----------------------------------
The possible events as I see are config file changes , commit /optimize. commit/optimize are already handled . so the main requirement is to make it possible for Solr to 'watch' a few files .
say
{code:java}
core.registerConfFileListener(listener, List<String> files);
{code}
this would need a new interface
{code:java}
public interface ConfFileListener {
public void confFileChanged(List<String> files);
}
{code}
> Provide a standard way to reload plugins
> ----------------------------------------
>
> Key: SOLR-1307
> URL: https://issues.apache.org/jira/browse/SOLR-1307
> Project: Solr
> Issue Type: New Feature
> Components: search, update
> Reporter: Shalin Shekhar Mangar
> Fix For: 1.5
>
>
> Currently, Solr plugins have no standard way to reload themselves. Each plugin invents its own mechanism e.g. SpellCheckComponent. For others, even small changes to configuration files are visible only after a core reload. Examples include changing elevate.xml, stopwords.txt etc.
> We should provide a standard way for plugins to reload themselves on events relevant to them.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.