[jira] Issue Comment Edited: (SOLR-561) Solr replication by Solr (for windows also)

["Noble Paul (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611989#action_12611989 ] 

noble.paul edited comment on SOLR-561 at 7/9/08 4:31 AM:
---------------------------------------------------------

This patch relies on the IndexDeletionPolicy to identify files to be replicated. It also supposrts replication of conf files. No need to register any listeners/ QueryResponseWriters

The configuration is as follows
on master 
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
        <!--Replicate on 'optimize' it can also be  'commit' -->
        <str name="replicateAfter">commit</str>
        <!--Config files to be to be replicated-->
         <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>          
    </lst>
</requestHandler>
{code}
on slave
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="slave">
        <str name="masterUrl">http://localhost:port/solr/corename/replication</str>  
        <str name="pollInterval">00:00:20</str>  
     </lst>
</requestHandler>
{code}

The Replication strategy is changed as follows
* CMD_INDEX_VERSION: gets the version of the current _IndexCommit_ to be replicated from the master. if the version is same, no need to replicate. If it is different
* CMD_FILE_LIST : Get the list of file names for the current _IndexCommit_ . Checks with the local index and identifies modified files by comparing names an sizes. 
* CMD-FILE_CONTENT : For each files to be downloaded, issue this command an download the content to a temp folder. After successful completion copy them to the index folder  and isse a commit
* If the current index is stale, or not able to synchronize, copy all the files . An index,properties file is written, which has the location of the new index directory
* _CoreDescriptor_ has a new method to reload core.
* If conf files are modified they are copied to the conf folder after taking a backup of the old. Then the core is reloaded








      was (Author: noble.paul):
    This patch relies on the IndexDeletionPolicy to identify files to be replicated. It also supposrts replication of conf files.

The configuration is as follows
on master 
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
        <!--Replicate on 'optimize' it can also be  'commit' -->
        <str name="replicateAfter">commit</str>
        <!--Config files to be to be replicated-->
         <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>          
    </lst>
</requestHandler>
{code}
on slave
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="slave">
        <str name="masterUrl">http://localhost:port/solr/corename/replication</str>  
        <str name="pollInterval">00:00:20</str>  
     </lst>
</requestHandler>
{code}

The Replication strategy is changed as follows
* CMD_INDEX_VERSION: gets the version of the current _IndexCommit _ to be replicated from the master. if the version is same, no need to replicate. If it is different
* CMD_FILE_LIST : Get the list of file names for the current _IndexCommit_ . Checks with the local index and identifies modified files by comparing names an sizes. 
* CMD-FILE_CONTENT : For each files to be downloaded, issue this command an download the content to a temp folder. After successful completion copy them to the index folder  and isse a commit
* If the current index is stale, or not able to synchronize, copy all the files . An index,properties file is written, which has the location of the new index directory
* _CoreDescriptor_ has a new method to reload core.
* If conf files are modified they are copied to the conf folder after taking a backup of the old. Then the core is reloaded







  
> Solr replication by Solr (for windows also)
> -------------------------------------------
>
>                 Key: SOLR-561
>                 URL: https://issues.apache.org/jira/browse/SOLR-561
>             Project: Solr
>          Issue Type: New Feature
>          Components: replication
>    Affects Versions: 1.3
>         Environment: All
>            Reporter: Noble Paul
>         Attachments: deletion_policy.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch
>
>
> The current replication strategy in solr involves shell scripts . The following are the drawbacks with the approach
> *  It does not work with windows
> * Replication works as a separate piece not integrated with solr.
> * Cannot control replication from solr admin/JMX
> * Each operation requires manual telnet to the host
> Doing the replication in java has the following advantages
> * Platform independence
> * Manual steps can be completely eliminated. Everything can be driven from solrconfig.xml .
> ** Adding the url of the master in the slaves should be good enough to enable replication. Other things like frequency of
> snapshoot/snappull can also be configured . All other information can be automatically obtained.
> * Start/stop can be triggered from solr/admin or JMX
> * Can get the status/progress while replication is going on. It can also abort an ongoing replication
> * No need to have a login into the machine 
> This issue can track the implementation of solr replication in java

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.