You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "John Wang (JIRA)" <ji...@apache.org> on 2007/05/21 08:43:16 UTC
[jira] Created: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Create a hook to allow custome code to create custome index readers
-------------------------------------------------------------------
Key: SOLR-243
URL: https://issues.apache.org/jira/browse/SOLR-243
Project: Solr
Issue Type: Improvement
Components: search
Affects Versions: 1.3
Environment: Solr core
Reporter: John Wang
Fix For: 1.3
I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated SOLR-243:
---------------------------
Attachment: indexReaderFactory.patch
My apologies for not being patient with this process.
I have made the requested changes and submitted another patch.
Please let me know if these are the correct things to do.
Thanks
-John
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
Okay, I am after SOLR-465, but since this guy hovers around it, so:
New patch with a test to be sure the config / alt factory is indeed used. Factory class loaded in SolrCore now with the pluginloader - added a new loadSingle method. SolrIndexConfig really doesn't seem like the right place for that method that was there anyway...
So, in pushing SOLR-465, I'd like to do things pretty much as is over there (except use the pluginloader stuff) just before loading the reader factory, so that the correct Directory imp can be passed to the getReader method. So comments to how this is done here will direct what I attempt there.
Thoughts? Pick away please, I'd like to help out wrapping this and 465 soon.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602844#action_12602844 ]
John Wang commented on SOLR-243:
--------------------------------
Not sure what do you mean?It was supposed to be a simple addition.
-John
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503959 ]
Hoss Man commented on SOLR-243:
-------------------------------
1) i'm sorry, i transposed the lines in my mind when i was readingthe patch (you've made a private constructor public, not the otherway arround -- my mistake)
2) yes, you're using Config.findClass ... what yonik asked was if there was a particular reason not to use Config.newInstance(name) in the loadIndexReaderFactory ... there is a lot of duplicate code in that method (mainly exception handling) that Config.newInstance takes care of for you.
3) I think you're missing my point about indexDefaults and mainIndex ... it's not a matter of just picking one, it's making it work with both so that a factory can be specified in the defaults for use anytime an IndexReader is opened, or from mainIndex awhen the "main index" is opened. I just poked around and found that the relevant class is "SolrIndexConfig" ... my suggestion was that this be where the IndexReaderFactory hook be so that it works the same way.
I'm sorry if you feel like you are jumping through a lot of hoops ... it's not my intention to be difficult, i'm just making comments on the patch and asking general questions (not specificly directed at your patch) about how Solr as a project can best support the topic of this issue (hooks to allow custom code to create custom index readers).
If the patch you have works well for you that's great, but that doesn't mean it will work well for everyone, which is something committers have to keep that in mind ... making public API changes (including new config syntax and especially new plugin hooks) is a serious change to the project and has to be considered very carefully because we have to be able to support it for a very very long time.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
Updates patch to trunk
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Affects Version/s: (was: 1.3)
Fix Version/s: 1.3
Assignee: Hoss Man
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic updated SOLR-243:
----------------------------------
Fix Version/s: 1.4
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500217 ]
Yonik Seeley commented on SOLR-243:
-----------------------------------
Thanks John, I also have a need for this type of functionality.
In my case, I need to open open a MultiReader over several index segments.
One thing that crosses my mind is if some factories will need a little more context, such as what reader is being created (for what purpose). For example, is it the main index reader, or is it for something like the spelling index, is it only for deleting in the IndexWriter, etc.
When instantiating the factory, why not just call Config.newInstance(name) and let it throw an exception (or did you want to continue on after a failure to find or instantiate?)
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Attachment: indexReaderFactory.patch
thinking about the legacy SolrIndexSearch constructors a bit more, and how SolrIndexConfig is used, i realized i few things...
* we can just always use the mainIndex config in those constructors
* we should really deprecate one of those constructors anyway
* we don't need to be quite as paranoid about getting the defaultIndexConfig if main isn't set -- SolrIndexConfig takes care of that for us.
* there were some tabs still left in the diff.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503697 ]
John Wang commented on SOLR-243:
--------------------------------
Are you looking at the right patch?
For 1) at least, they are public.
2) I am using Config.findClass
3) I just picked one from your suggestions, mainIndex is fine.
I feel I am jumping through quite a few arbitrary hoops just to contribute to an open source project, and this is wrong and defeats the whole purpose of the idea of open source.
I think being more constructive and result oriented instead of being difficult and purist can benefit this project much more.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated SOLR-243:
---------------------------
Attachment: indexReaderFactory.patch
New patch incorporating Chris's comments 1-6.
for item 3) config is now placed under indexDefaults section.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12576693#action_12576693 ]
John Wang commented on SOLR-243:
--------------------------------
What is the plan for this patch?
We have a couple of votes for it.
Thanks
-John
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12549654 ]
John Wang commented on SOLR-243:
--------------------------------
Thanks Kevin!
Please vote on this patch to have it committed.
-John
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500261 ]
John Wang commented on SOLR-243:
--------------------------------
Hi Yonik:
1) context in factory: I am not sure how the interface would look like, suggestions?
2) I catch the exception to default to the normal IndexReader.open mechanism.
3) I just picked it to be an interface, I think making it an abstract class is fine.
thanks
-John
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
to trunk
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12578553#action_12578553 ]
Hoss Man commented on SOLR-243:
-------------------------------
John: first off, i want to apologize for the *extremely* belated patch review ... I know i told you i'd try to look at this months ago, but ... yeah ... life tends to get in the way of patches.
on the whole, things seem pretty good -- although you were still using an interface instead of an abstract class. Having the main API for a type of plugin be an abstract class is an important mechanism to help future proof ourselves against possible additions we want to make the the plugin API in the future; if it's an interface we can't really change it once it's released (because we might break people Impls); if it's an abstract class, we can always provide a default impl for people.
the biggest concern i have about this patch at the moment is that there are no tests, and no example configs, so it's hard to be sure it's even working at all :)
here's what i see as the current todo list for this issue...
1) there are some legacy SolrIndexSearcher constructors that need to delegate to the SolrCore to get indexReaderFactory ... perhaps we should have a helper method that decides which SolrIndexConfig to use based on the "name" ?
2) we need tests showing custom IndexReaderFactory getting used (even if it's just a mock IndexReaderFactory thatsets a boolean to show it's being used) ... this will also serve as a test that config syntax works.
3) need commented out example of using a custom indexReaderFactory in example/solr/conf/solrconfig.xml
4) sanity check this against SOLR-465, make sure we aren't painting ourselves into a corner.
5) we should make IndexReaderFactory use the AbstractPluginLoader stuff and remove the guts of SolrIndexConfig.loadIndexReaderFactory. It looks like we'll need to add a "single item" version of load to the AbstractPluginLoader to make that work well.
#5 is something that can be done after this is committed, but 1-4 are pretty important. If you can update the patch with some configs/tests i'll think about the legacy SolrIndexSearcher constructors and try to figure out a good solution for them
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Attachment: indexReaderFactory.patch
Grrr.... reattaching with the right radio button this time.
further refinement: made IndexReaderFactory an abstract class extending NamedListInitializedPlugin.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
I'm going to commit this if there are no objections.
- Mark
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated SOLR-243:
---------------------------
Attachment: indexReaderFactory.patch
I have attached a patch for this issue.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713024#action_12713024 ]
Mark Miller commented on SOLR-243:
----------------------------------
Yup I know - in the case that that goes in first, its easier to resolve (I had already put it here). And if it does not, I'll remove it before committing.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713022#action_12713022 ]
Shalin Shekhar Mangar commented on SOLR-243:
--------------------------------------------
Mark, this patch still has the code for SOLR-1184.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
to trunk
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605447#action_12605447 ]
John Wang commented on SOLR-243:
--------------------------------
Sorry, I didn't see Hoss's earlier comments.
Thanks
-John
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrzej Bialecki updated SOLR-243:
-----------------------------------
Attachment: SOLR-243.patch
This is a useful functionality when using FilterIndexReader or ParallelReader, +1 for adding it to core. I updated the patch to the latest trunk - all tests pass.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas updated SOLR-243:
----------------------------
Fix Version/s: (was: 1.3)
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500262 ]
John Wang commented on SOLR-243:
--------------------------------
My motivation in doing this is to create a plugin into solr for my project:
www.browseengine.com
or
bobo-browse.sourceforge.net
which is a more extensive facetted search implementation. I need a special IndexReader to do the plugin.
Please take a look.
Thanks
-John
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
Adds a config option to turn on/off reopening readers. apache pork.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602860#action_12602860 ]
Mike Klaas commented on SOLR-243:
---------------------------------
Hi John,
Hoss has marked the issue for 1.3, so it will be in the release.
-Mike
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
to trunk, remove non readonly reader factory method and fixes readOnly propagation in a couple spots.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Kevin Osborn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12549609 ]
Kevin Osborn commented on SOLR-243:
-----------------------------------
We used this patch with great success. We have a custom multiindex reader. This patch allowed us to just plug in our changes without resorting to hack up the base Solr code.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic updated SOLR-243:
----------------------------------
Summary: Create a hook to allow custom code to create custom IndexReaders (was: Create a hook to allow custome code to create custome index readers)
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated SOLR-243:
---------------------------
Description:
I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
IndexReader.open(path)
It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
interface IndexReaderFactory{
IndexReader newReader(String name,String path);
}
the default implementation would just return: IndexReader.open(path)
And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
Thanks
-John
was:
I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499294 ]
John Wang commented on SOLR-243:
--------------------------------
1) Sorry about this, my eclipse IDE showed 2 spaces. I will remember in the future.
2) Sorry about this one as well, I changed them from public to private to do some testing and forgot to change them back. I have no problem with them being public.
3) That sounds good. I just picked a spot in solrConfig.xml
4) Sure.
5) I think that is a coding style, but I am not religious about it.
6) That is a great idea! I wanted to add this but wasn't sure how to do it.
7) Ok.
Thanks for the code review!
-John
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Attachment: indexReaderFactory.patch
minimal set of changes i could make to previous patch so that: a) it compiles against the trunk r636903; b) it doesn't rely on the deprecated singletons (SolrConfig or SolrCore)
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller resolved SOLR-243.
------------------------------
Resolution: Fixed
Fix Version/s: (was: 1.5)
1.4
Committed, thanks to all involved.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503691 ]
Hoss Man commented on SOLR-243:
-------------------------------
comments on the latest patch..
1) SolrIndexSeacher constructor is still private
2) i believe Yonik's question about calling Config.newInstance(name) was in realtion to initializing the Factory (you currently just use "factory = clazz.newInstance()")
3) my comment about the mainIndex/indexDefaults blocks was that for many options they can be specified in the indexDefaults section, and overriden in the mainIndex section ... from what i can tell the current patch requires it to be in indexDefaults, ideally it would follow the same behavior as the other settings (mainIndex is used for the .. well, "main index" while indexDefaults also applies to other intermediate indexes ... down the road we may deprecate it, or expand it's usage for other types of micro indexes various pieces of functionality use like spellchecking) but for now any option relating to an indexreader/writer should support the "dfaulting mechanism and work with both sections.
(I'm not sure what type of API Yonik might have been thinking of for conveying context).
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500219 ]
Yonik Seeley commented on SOLR-243:
-----------------------------------
Minor nit: is there a reason that IndexReaderFactory can't be a class instead of an interface?
It doesn't seem to be a likely candidate for multiple inheritance, and making it a class allows us to "upgrade" the interface in the future while providing a backward compatible default implementation for people who have already implemented it.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12605424#action_12605424 ]
Hoss Man commented on SOLR-243:
-------------------------------
bq. Hoss has marked the issue for 1.3, so it will be in the release.
for the record, i marked it as 1.3 because itwould be nice to see this in 1.3 ... but as i said in my 2008-03-13 comment: we need unit tests and example configuration before i'm willing to commit.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas updated SOLR-243:
----------------------------
Do we still want to target 1.3 here? (Seems like there is a lot to do before it is commit-worthy, based on the comments)
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Attachment: indexReaderFactory.patch
further refinement: made IndexReaderFactory an abstract class extending NamedListInitializedPlugin.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
Updated to trunk and now uses alt directory test config file rather than a new one.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499369 ]
Hoss Man commented on SOLR-243:
-------------------------------
regarding the IndexReaderFactory.DEFAULT, i have ahard time viewing it as good coding style at all ..but then again i've never seen any literature arguming for or against the concept, so i'm not sure what the "pros" are. My personal list of "cons"...
a) violates the API / impl abstraction; interfaces shouldn't have to know about any of their Impls
b) forces people who want to write their own impl to have the "default" class loaded at run time even if they don't use it.
c) can't be changed; static variables in interfaces are be final (even if they aren't declared final) and can give the false impression to people reading an API that they can change it to something else.
as far as #6 above (an init method) SolrCore.parseListener is a good example of how to NamedList style initing (see the SolrEventListener API). IndexSchema.readTokenFilterFactory is a good example of the simpler Map style initing.
I saw an idea thrown around recently about making super interfaces for each of the two types of initialization so a lot of this code could be refactored away ... but that probably won't happen soon, and when/if it does it will be a big undertaking that will clean up everything, so in the mean time a cut/paste/tweak approach would be the best way to proceed
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: Parititoning support ?
Posted by Chris Hostetter <ho...@fucit.org>.
: Now we want to move to SOLR as it has host of other features. I was
: reading http://wiki.apache.org/solr/IndexPartitioning .
:
: Is there any JIRA issue corresponding to this feature (Tried searching
: but couldn't find any) ?
not directly, that was mainly just a brainstorming excercise. there is a
patch that tackles the related goal of multiple "SolrCores" in a single
app context however...
https://issues.apache.org/jira/browse/SOLR-215
...but i havne't relaly had a chance to look at it yet.
honestly, based on the description of your goal, it doens't sound like
"IndexPartitioning" is really what you want (the goal there is multiple
sub indexes managed by a single instance of solr) ... it sounds like what
you are interested in is more along the lines of...
http://wiki.apache.org/solr/FederatedSearch
...multiple machines each with a seperate "shareds" of the index.
-Hoss
Parititoning support ?
Posted by Sharad Agarwal <sh...@aol.com>.
I am looking for the partitioning support in Solr. I have huge corpus of
documents ~ 100M; so partitioning the index is almost a necessity.
Currently we are using our internal developed index/search server based
on lucene; and have implemented partitioning. It exposes Remote
Searchable and work over Multisearcher.
Now we want to move to SOLR as it has host of other features. I was
reading http://wiki.apache.org/solr/IndexPartitioning .
Is there any JIRA issue corresponding to this feature (Tried searching
but couldn't find any) ?
Sharad Agarwal
Search Platforms Group
AOL
[jira] Commented: (SOLR-243) Create a hook to allow custome code to
create custome index readers
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499284 ]
Hoss Man commented on SOLR-243:
-------------------------------
some comments after reading the patch (haven't run it yet) ...
1) in the future, please use 2 spaces for indenting (and no tabs)
2) the existing public constructors for SolrIndexSearcher are now private which is an API change that has to be carefully considered ... on top of that, they don't even use the new IndexReder factory at all (they could get it by asking the SolrCore - it's a singleton).
3) instead of adding a new indexReaderFactory element directly to the solrconfig.xml, it would probably make sense to get it when parsing the mainIndex/indexDefaults blocks.
4) StandardIndexReaderFactory" might be a better class name then "DefaultIndexReaderFactory"
5) I don't think we really need IndexReaderFactory.DEFAULT (static final instances in interfaces never make sense to me, they are not part of the "interface") ... just let SolrCore have a hardcoded instance of to use if it can't find one in the config.
6) people should be able to specify configuration options for their factories ... either using an init(NamedLIst) method (like RequestHandlers) or using an init(Map<String,String>) method (like Caches, TokenFilters, etc...)
7) catching "Exception" when calling newInstance() is too broad. explicitly catch only the exceptions that are expected and warrant using the default factory, otherwise you might silently ignore a really serious problem. ... although frankly, if someone configures an explict IndexReader, and it can't be instantiated, that should probably be SEVERE not just WARNING.
> Create a hook to allow custome code to create custome index readers
> -------------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Environment: Solr core
> Reporter: John Wang
> Fix For: 1.3
>
> Attachments: indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reassigned SOLR-243:
--------------------------------
Assignee: Mark Miller (was: Hoss Man)
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Mark Miller
> Fix For: 1.5
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670666#action_12670666 ]
Shalin Shekhar Mangar commented on SOLR-243:
--------------------------------------------
Mark, are you still interested in this issue? If not, should we mark this to 1.5?
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-243:
--------------------------
Fix Version/s: (was: 1.3)
unmarking for 1.3 ... on hold until we have tests and/or more interest
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-243) Create a hook to allow custom code to
create custom IndexReaders
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-243:
-----------------------------
Attachment: SOLR-243.patch
First attempt at getting this back up to trunk.
Should probably merge the alt dir and alt reader test config files into one.
> Create a hook to allow custom code to create custom IndexReaders
> ----------------------------------------------------------------
>
> Key: SOLR-243
> URL: https://issues.apache.org/jira/browse/SOLR-243
> Project: Solr
> Issue Type: Improvement
> Components: search
> Environment: Solr core
> Reporter: John Wang
> Assignee: Hoss Man
> Fix For: 1.4
>
> Attachments: indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, indexReaderFactory.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch, SOLR-243.patch
>
>
> I have a customized IndexReader and I want to write a Solr plugin to use my derived IndexReader implementation. Currently IndexReader instantiation is hard coded to be:
> IndexReader.open(path)
> It would be really useful if this is done thru a plugable factory that can be configured, e.g. IndexReaderFactory
> interface IndexReaderFactory{
> IndexReader newReader(String name,String path);
> }
> the default implementation would just return: IndexReader.open(path)
> And in the newSearcher and getSearcher methods in SolrCore class can call the current factory implementation to get the IndexReader instance and then build the SolrIndexSearcher by passing in the reader.
> It would be really nice to add this improvement soon (This seems to be a trivial addition) as our project really depends on this.
> Thanks
> -John
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.