You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Jianhan (JIRA)" <ji...@apache.org> on 2009/05/19 18:44:45 UTC
[jira] Created: (SOLR-1175) disable replication on master side
disable replication on master side
----------------------------------
Key: SOLR-1175
URL: https://issues.apache.org/jira/browse/SOLR-1175
Project: Solr
Issue Type: New Feature
Components: replication (java)
Affects Versions: 1.4
Environment: any
Reporter: Jianhan
Priority: Minor
Fix For: 1.4
In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed.
Hi,
Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it
1. on all slaves: disable replication
2. on master: stop the server
3. on master: delete all the documents
4. on master: restart the server
5. on master: index all documents
6. on slaves: enable replication
The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only.
Is that possible?
By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4.
Thanks,
Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Resolved: (SOLR-1175) disable replication on master side
Posted by Jian Han Guo <ji...@gmail.com>.
That is great, and I tried it and it looks working great.
Now how do we disable/enable replication programmatically?
Here is what I tried:
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("qt", "/replication");
params.set("command", "disablereplication");
solrServer.query(params);
where solrServer is an instance of SolrServer.
It seems working fine (meaning I didn't get any exceptions). The questions
are:
1. is this the right way to do it? It looks odd to call the function query
to disable replication, but I don't find a better way to do it.
SolrServer.request seems a better choice, but I don't know which subclass of
SolrRequest to use and the comments of the class indicates these classes may
change.
2. how can we verify that replication is actually disabled/enabled?
launching a slave and starting replication is a way to verify, but it would
be nice if we can see the status in replication details page.
Thanks,
Jianhan
On Wed, May 20, 2009 at 11:58 PM, Noble Paul (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Noble Paul resolved SOLR-1175.
> ------------------------------
>
> Resolution: Fixed
>
> committed revision: 776978
>
> > disable replication on master side
> > ----------------------------------
> >
> > Key: SOLR-1175
> > URL: https://issues.apache.org/jira/browse/SOLR-1175
> > Project: Solr
> > Issue Type: New Feature
> > Components: replication (java)
> > Affects Versions: 1.4
> > Environment: any
> > Reporter: Jianhan
> > Assignee: Noble Paul
> > Priority: Minor
> > Fix For: 1.4
> >
> > Attachments: SOLR-1175.patch
> >
> > Original Estimate: 4h
> > Remaining Estimate: 4h
> >
> > In an environment where one master and several slaves are deployed, it
> usually takes more effort to discover all the slaves and disable replication
> on slave side (which is available right now), and it would be much easier to
> disable it on master instance (when, for example, there is a need to rebuild
> the index, yet search has to continue). The following is the original email
> describing a scenario when this feature is needed.
> > Hi,
> > Occasionally, we want to build our indexes from scratch, and during this
> period we want our search continue to work. Here are the steps that I think
> will do it
> > 1. on all slaves: disable replication
> > 2. on master: stop the server
> > 3. on master: delete all the documents
> > 4. on master: restart the server
> > 5. on master: index all documents
> > 6. on slaves: enable replication
> > The only problem is: step 1 and 6. We may schedule any time to rebuild
> indexes and it is an automated process. It is possible to let the master to
> disable replication on all slaves, but then we have to discover all the
> slaves automatically, also exceptions may happen, e.g. a slave may be down
> at the time and then restarted later on. Anyhow it becomes an unreliable
> process. So I am thinking of somehow disabling replication on the master
> side during reindex, i.e. set a state on master so that any request for
> replication will be ignored. That all the steps described above will be on
> master side only.
> > Is that possible?
> > By the way, I am talking about solr 1.4. I looked at how 1.3 works, and
> noticed that in 1.3 there is a way to disable replication on master side:
> shutdown rsyncd, so I guess it would be nice to have something equivalent in
> solr 1.4.
> > Thanks,
> > Jianhan
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
[jira] Commented: (SOLR-1175) disable replication on master side
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711492#action_12711492 ]
Noble Paul commented on SOLR-1175:
----------------------------------
two commands are added 'disableReplication' and 'enableReplication'
> disable replication on master side
> ----------------------------------
>
> Key: SOLR-1175
> URL: https://issues.apache.org/jira/browse/SOLR-1175
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
> Affects Versions: 1.4
> Environment: any
> Reporter: Jianhan
> Assignee: Noble Paul
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1175.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed.
> Hi,
> Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it
> 1. on all slaves: disable replication
> 2. on master: stop the server
> 3. on master: delete all the documents
> 4. on master: restart the server
> 5. on master: index all documents
> 6. on slaves: enable replication
> The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only.
> Is that possible?
> By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4.
> Thanks,
> Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1175) disable replication on master side
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noble Paul reassigned SOLR-1175:
--------------------------------
Assignee: Noble Paul
> disable replication on master side
> ----------------------------------
>
> Key: SOLR-1175
> URL: https://issues.apache.org/jira/browse/SOLR-1175
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
> Affects Versions: 1.4
> Environment: any
> Reporter: Jianhan
> Assignee: Noble Paul
> Priority: Minor
> Fix For: 1.4
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed.
> Hi,
> Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it
> 1. on all slaves: disable replication
> 2. on master: stop the server
> 3. on master: delete all the documents
> 4. on master: restart the server
> 5. on master: index all documents
> 6. on slaves: enable replication
> The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only.
> Is that possible?
> By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4.
> Thanks,
> Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1175) disable replication on master side
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noble Paul updated SOLR-1175:
-----------------------------
Attachment: SOLR-1175.patch
> disable replication on master side
> ----------------------------------
>
> Key: SOLR-1175
> URL: https://issues.apache.org/jira/browse/SOLR-1175
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
> Affects Versions: 1.4
> Environment: any
> Reporter: Jianhan
> Assignee: Noble Paul
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1175.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed.
> Hi,
> Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it
> 1. on all slaves: disable replication
> 2. on master: stop the server
> 3. on master: delete all the documents
> 4. on master: restart the server
> 5. on master: index all documents
> 6. on slaves: enable replication
> The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only.
> Is that possible?
> By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4.
> Thanks,
> Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1175) disable replication on master side
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Noble Paul resolved SOLR-1175.
------------------------------
Resolution: Fixed
committed revision: 776978
> disable replication on master side
> ----------------------------------
>
> Key: SOLR-1175
> URL: https://issues.apache.org/jira/browse/SOLR-1175
> Project: Solr
> Issue Type: New Feature
> Components: replication (java)
> Affects Versions: 1.4
> Environment: any
> Reporter: Jianhan
> Assignee: Noble Paul
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-1175.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> In an environment where one master and several slaves are deployed, it usually takes more effort to discover all the slaves and disable replication on slave side (which is available right now), and it would be much easier to disable it on master instance (when, for example, there is a need to rebuild the index, yet search has to continue). The following is the original email describing a scenario when this feature is needed.
> Hi,
> Occasionally, we want to build our indexes from scratch, and during this period we want our search continue to work. Here are the steps that I think will do it
> 1. on all slaves: disable replication
> 2. on master: stop the server
> 3. on master: delete all the documents
> 4. on master: restart the server
> 5. on master: index all documents
> 6. on slaves: enable replication
> The only problem is: step 1 and 6. We may schedule any time to rebuild indexes and it is an automated process. It is possible to let the master to disable replication on all slaves, but then we have to discover all the slaves automatically, also exceptions may happen, e.g. a slave may be down at the time and then restarted later on. Anyhow it becomes an unreliable process. So I am thinking of somehow disabling replication on the master side during reindex, i.e. set a state on master so that any request for replication will be ignored. That all the steps described above will be on master side only.
> Is that possible?
> By the way, I am talking about solr 1.4. I looked at how 1.3 works, and noticed that in 1.3 there is a way to disable replication on master side: shutdown rsyncd, so I guess it would be nice to have something equivalent in solr 1.4.
> Thanks,
> Jianhan
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.