[jira] Commented: (SOLR-255) RemoteSearchable for Solr(use RMI)

["Toru Matsuzawa (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12510631 ] 

Toru Matsuzawa commented on SOLR-255:
-------------------------------------

Hi Yonik,

SolrMultiSearcher execute searches in each index file.
Index files to be searched is as follows:
 - /data/index (default index files, read/write)
 - local index files (read only)
 - remote index files through RMI (read only)

SolrIndexSearchable performs searches in local indexes, 
which originally SolrIndexSearcher performs.
SolrClientSearchable is executed as the wrapper instance 
which gets a RMI object and performs searches 

In SolrMultiSearcher, SolrFieldSortedHitQueue merged search results 
from all searched indexes.
(FieldSortedHitQueue cannot use here 
because it requires using FieldCache.)

When RMI is valid, the following data are transferred through the network.
1) Weight, DocSet, Sort and so on, for search requests to remote indexes
2) DocListAndSet , DocList or DocSet as these search results
3) FieldCache of each field from remote indexes

With getInts(field), getFloats(field) and so on, 
SolrMultiSearcher merges all these data, 
which SolrIndexSearchable or SolrClientSearchable for each index gets 
with getInts(field), getFloats(field) and so on.
There will be certain problems on scalability unless something should be done for this point.（Please teach if there is something a good idea. )

Each index is searched sequentially in my present implementation.
This process should be in parallel in future implementation. (It must be hard to implement.)

My implementation is mainly in order to search indexes on remote nodes through RMI.
Without RMI (i.e., local index searches only), my implementation can be more simple.
(I have originally implemented this for its remote index use in my project.
 However, I use Solr with local indexes only because of complex operations of remote index version.)

Thanks,

> RemoteSearchable for Solr(use RMI)
> ----------------------------------
>
>                 Key: SOLR-255
>                 URL: https://issues.apache.org/jira/browse/SOLR-255
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>            Reporter: Toru Matsuzawa
>         Attachments: solr-multi20070606.zip
>
>
> I experimentally implemented RemoteSearchable of Lucene for Solr.
> I referred to "FederatedSearch" and used RMI. 
> Two or more Searchers can be referred to with SolrIndexSearcher.
> These query-only indexes can be specified in solrconfig.xml, 
> enumerating the list under a <searchIndex> tag.
>   <searchIndex>
>     <lst>E:\sample\data1</lst>
>     <lst>E:\sample\data2</lst>
>     <lst>rmi://localhost</lst>
>   </searchIndex>
> The index in the dataDir is also used as the default index of solr
> to update and query.
> When data of a document in a index specified under the <searchIndex> is
> updated, 
> that document data in the index will be deleted and data of the updated document will be stored
> in the index in the dataDir.
> SolrRemoteSearchable (the searcher for remote access) is started from SolrCore 
> by specifying "< remoteSearcher>true</remoteSearcher >" in solrconfig.xml.(It is registered in RMI. )
> ("-Djava.security.policy" should be set when you start VM. )
> Not all of the operational cases are tested 
> because Solr has so many features. 
> Moreover, TestUnit has not been made 
> because I made this through a trial and error process. 
> Some changes are required in Lucene to execute this. 
> I need your comments on this although it might be hard without TestUnit. 
> I especially worry about the followings: 
> - Am I on the right truck about this issue?
> - Is the extent of modifying Lucene tolerable?
> - Are there any ideas to implement this feature without modifying Lucene?
> - Does this idea contribute for improving Solr?
> - This implementation may partially overlap with "Multiple Solr Cores".
>   What should be done?
> - Are there any other considerations about this issue, which I have overlooked?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.