You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mingfeng Yang <mf...@wisewindow.com> on 2013/01/26 01:42:48 UTC
How to migrate SolrCloud shards to different servers?
Right now I have an index with four shards on a single EC2 server, each
running on different ports. Now I'd like to migrate three shards
to independent servers.
What should I do to safely accomplish this process?
Can I just
1. shutdown all four solr instances.
2. copy three shards (indexes) to different servers.
3. launch 4 solr instances on 4 different servers, each with -zKhost
specified, pointing to the zookeeper servers.
In my impression, zookeeper remembers which shards are leaders. What I
plan to do above could not elect the three new servers as leaders. If so,
what's the correct way to do it?
Thanks,
Ming
Re: How to migrate SolrCloud shards to different servers?
Posted by Shawn Heisey <so...@elyograg.org>.
On 1/26/2013 11:37 AM, Mark Miller wrote:
> Hey Shawn - got a suggestion for an addition for the wiki that would
> have saved you some time here?
I have updated the SolrCloud wiki page. I put a small note under
Getting Started that references the meat of my addition under Parameter
Reference. Does it look OK?
I also added collection.configName to the parameter list for the
Collections API. I learned about this on IRC, and without it I would
not have been able to get a collection properly installed.
If there are any other parameters for either the Collections API or the
CoreAdmin API that users should know about, can someone please add them?
When I have some real time after getting my SolrCloud fully operational,
I will see what I can do to make the entire wiki page more
understandable for a beginner.
Thanks,
Shawn
Re: How to migrate SolrCloud shards to different servers?
Posted by Mark Miller <ma...@gmail.com>.
Hey Shawn - got a suggestion for an addition for the wiki that would
have saved you some time here?
- Mark
On Sat, Jan 26, 2013 at 1:22 PM, Shawn Heisey <so...@elyograg.org> wrote:
> On 1/26/2013 6:31 AM, Per Steffensen wrote:
>>
>> We have actually tested this and found that the following will do it
>> * Shutdown all Solr nodes - make sure ZKs are still running
>> * For each replica (shard-instance) move its data-folder to the new
>> server (if they are not already available to it through some shared
>> storage)
>> * For each repilca (shard-instance) also move solr.xmls
>> * Extract clusterstate.json from ZK into a file. Modify that file so
>> that hosts/IPs and ports are correct according to new setup. Replace
>> clusterstate.json in ZK with the modified content of the
>> clusterstate.json file
>> * Start new Solr nodes
>
>
> What I found is that you have to have the port correct in solr.xml.
>
> That is what is used when populating the "live_nodes" structure in ZK. When
> I first set up my little cloud, I changed the jetty port in jetty.xml to
> 8985, but found that live_nodes was getting populated with 8983.
>
> When I fixed jetty.xml to use the jetty.port system property (which is also
> in my solr.xml), and set that property at startup, live_nodes in ZK was
> correct with 8985 as the port.
>
> Thanks,
> Shawn
>
--
- Mark
Re: How to migrate SolrCloud shards to different servers?
Posted by Shawn Heisey <so...@elyograg.org>.
On 1/26/2013 6:31 AM, Per Steffensen wrote:
> We have actually tested this and found that the following will do it
> * Shutdown all Solr nodes - make sure ZKs are still running
> * For each replica (shard-instance) move its data-folder to the new
> server (if they are not already available to it through some shared
> storage)
> * For each repilca (shard-instance) also move solr.xmls
> * Extract clusterstate.json from ZK into a file. Modify that file so
> that hosts/IPs and ports are correct according to new setup. Replace
> clusterstate.json in ZK with the modified content of the
> clusterstate.json file
> * Start new Solr nodes
What I found is that you have to have the port correct in solr.xml.
That is what is used when populating the "live_nodes" structure in ZK.
When I first set up my little cloud, I changed the jetty port in
jetty.xml to 8985, but found that live_nodes was getting populated with
8983.
When I fixed jetty.xml to use the jetty.port system property (which is
also in my solr.xml), and set that property at startup, live_nodes in ZK
was correct with 8985 as the port.
Thanks,
Shawn
Re: How to migrate SolrCloud shards to different servers?
Posted by Timothy Potter <th...@gmail.com>.
Just one suggestion, instead of stopping zk and removing zoo_data,
better to use Solr's zkcli.sh script from cloud-scripts to clear out
data, e.g.
zkcli.sh -zkhost localhost:9983 -cmd clear /solr
The paths I clear when I want a full clean-up are:
/configs/CONFIG_NAME
/collections/COLLECTION_NAME
/clusterstate.json
On Tue, Jan 29, 2013 at 4:14 PM, Mingfeng Yang <mf...@wisewindow.com> wrote:
> An experiment found that stop all shards, remove the zoo_data (assume your
> zookeeper is used for this particular solrcloud, otherwise, be cautious),
> and then start instance by order works fine.
>
> Ming
>
>
>
> On Sat, Jan 26, 2013 at 5:31 AM, Per Steffensen <st...@designware.dk> wrote:
>
>> Hi
>>
>> We have actually tested this and found that the following will do it
>> * Shutdown all Solr nodes - make sure ZKs are still running
>> * For each replica (shard-instance) move its data-folder to the new server
>> (if they are not already available to it through some shared storage)
>> * For each repilca (shard-instance) also move solr.xmls
>> * Extract clusterstate.json from ZK into a file. Modify that file so that
>> hosts/IPs and ports are correct according to new setup. Replace
>> clusterstate.json in ZK with the modified content of the clusterstate.json
>> file
>> * Start new Solr nodes
>>
>> Good luck!
>>
>> Regards, Per Steffensen
>>
>>
>>
>> On 1/26/13 6:56 AM, Mingfeng Yang wrote:
>>
>>> Hi Mark,
>>>
>>> When I did testing with SolrCloud, I found the following.
>>>
>>> 1. I started 4 shards on the same host on port 8983, 8973, 8963, and 8953.
>>> 2. Index some data.
>>> 3. Shutdown all 4 shards.
>>> 4. Started 4 shards again, all pointing to the same data directory and use
>>> the same configuration, except that now we use different ports 8983, 8973,
>>> 7633 and 7648.
>>> 5. Now Solr has problem to load all cores properly.
>>>
>>> Therefore, I had the impression that ZooKeeper may have a memory of which
>>> hosts correspond to which shards. If I change the host info, it may get
>>> confused. I could not find any related documentation or discussion about
>>> this issue.
>>>
>>> Thanks,
>>> Ming
>>>
>>>
>>>
>>>
>>> On Fri, Jan 25, 2013 at 5:52 PM, Mark Miller <ma...@gmail.com>
>>> wrote:
>>>
>>> You could do it that way.
>>>>
>>>> I'm not sure why you are worried about the leaders. That shouldn't
>>>> matter.
>>>>
>>>> You could also start up new Solrs on the new machines as replicas of the
>>>> cores you want to move - then once they are active, unload the cores on
>>>> the
>>>> old machine, stop the Solr instances and remove the stuff left on the
>>>> filesystem.
>>>>
>>>> - Mark
>>>>
>>>> On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com>
>>>> wrote:
>>>>
>>>> Right now I have an index with four shards on a single EC2 server, each
>>>>> running on different ports. Now I'd like to migrate three shards
>>>>> to independent servers.
>>>>>
>>>>> What should I do to safely accomplish this process?
>>>>>
>>>>> Can I just
>>>>> 1. shutdown all four solr instances.
>>>>> 2. copy three shards (indexes) to different servers.
>>>>> 3. launch 4 solr instances on 4 different servers, each with -zKhost
>>>>> specified, pointing to the zookeeper servers.
>>>>>
>>>>> In my impression, zookeeper remembers which shards are leaders. What I
>>>>> plan to do above could not elect the three new servers as leaders. If
>>>>>
>>>> so,
>>>>
>>>>> what's the correct way to do it?
>>>>>
>>>>> Thanks,
>>>>> Ming
>>>>>
>>>>
>>>>
>>
Re: How to migrate SolrCloud shards to different servers?
Posted by Mingfeng Yang <mf...@wisewindow.com>.
An experiment found that stop all shards, remove the zoo_data (assume your
zookeeper is used for this particular solrcloud, otherwise, be cautious),
and then start instance by order works fine.
Ming
On Sat, Jan 26, 2013 at 5:31 AM, Per Steffensen <st...@designware.dk> wrote:
> Hi
>
> We have actually tested this and found that the following will do it
> * Shutdown all Solr nodes - make sure ZKs are still running
> * For each replica (shard-instance) move its data-folder to the new server
> (if they are not already available to it through some shared storage)
> * For each repilca (shard-instance) also move solr.xmls
> * Extract clusterstate.json from ZK into a file. Modify that file so that
> hosts/IPs and ports are correct according to new setup. Replace
> clusterstate.json in ZK with the modified content of the clusterstate.json
> file
> * Start new Solr nodes
>
> Good luck!
>
> Regards, Per Steffensen
>
>
>
> On 1/26/13 6:56 AM, Mingfeng Yang wrote:
>
>> Hi Mark,
>>
>> When I did testing with SolrCloud, I found the following.
>>
>> 1. I started 4 shards on the same host on port 8983, 8973, 8963, and 8953.
>> 2. Index some data.
>> 3. Shutdown all 4 shards.
>> 4. Started 4 shards again, all pointing to the same data directory and use
>> the same configuration, except that now we use different ports 8983, 8973,
>> 7633 and 7648.
>> 5. Now Solr has problem to load all cores properly.
>>
>> Therefore, I had the impression that ZooKeeper may have a memory of which
>> hosts correspond to which shards. If I change the host info, it may get
>> confused. I could not find any related documentation or discussion about
>> this issue.
>>
>> Thanks,
>> Ming
>>
>>
>>
>>
>> On Fri, Jan 25, 2013 at 5:52 PM, Mark Miller <ma...@gmail.com>
>> wrote:
>>
>> You could do it that way.
>>>
>>> I'm not sure why you are worried about the leaders. That shouldn't
>>> matter.
>>>
>>> You could also start up new Solrs on the new machines as replicas of the
>>> cores you want to move - then once they are active, unload the cores on
>>> the
>>> old machine, stop the Solr instances and remove the stuff left on the
>>> filesystem.
>>>
>>> - Mark
>>>
>>> On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com>
>>> wrote:
>>>
>>> Right now I have an index with four shards on a single EC2 server, each
>>>> running on different ports. Now I'd like to migrate three shards
>>>> to independent servers.
>>>>
>>>> What should I do to safely accomplish this process?
>>>>
>>>> Can I just
>>>> 1. shutdown all four solr instances.
>>>> 2. copy three shards (indexes) to different servers.
>>>> 3. launch 4 solr instances on 4 different servers, each with -zKhost
>>>> specified, pointing to the zookeeper servers.
>>>>
>>>> In my impression, zookeeper remembers which shards are leaders. What I
>>>> plan to do above could not elect the three new servers as leaders. If
>>>>
>>> so,
>>>
>>>> what's the correct way to do it?
>>>>
>>>> Thanks,
>>>> Ming
>>>>
>>>
>>>
>
Re: How to migrate SolrCloud shards to different servers?
Posted by Per Steffensen <st...@designware.dk>.
Hi
We have actually tested this and found that the following will do it
* Shutdown all Solr nodes - make sure ZKs are still running
* For each replica (shard-instance) move its data-folder to the new
server (if they are not already available to it through some shared storage)
* For each repilca (shard-instance) also move solr.xmls
* Extract clusterstate.json from ZK into a file. Modify that file so
that hosts/IPs and ports are correct according to new setup. Replace
clusterstate.json in ZK with the modified content of the
clusterstate.json file
* Start new Solr nodes
Good luck!
Regards, Per Steffensen
On 1/26/13 6:56 AM, Mingfeng Yang wrote:
> Hi Mark,
>
> When I did testing with SolrCloud, I found the following.
>
> 1. I started 4 shards on the same host on port 8983, 8973, 8963, and 8953.
> 2. Index some data.
> 3. Shutdown all 4 shards.
> 4. Started 4 shards again, all pointing to the same data directory and use
> the same configuration, except that now we use different ports 8983, 8973,
> 7633 and 7648.
> 5. Now Solr has problem to load all cores properly.
>
> Therefore, I had the impression that ZooKeeper may have a memory of which
> hosts correspond to which shards. If I change the host info, it may get
> confused. I could not find any related documentation or discussion about
> this issue.
>
> Thanks,
> Ming
>
>
>
>
> On Fri, Jan 25, 2013 at 5:52 PM, Mark Miller <ma...@gmail.com> wrote:
>
>> You could do it that way.
>>
>> I'm not sure why you are worried about the leaders. That shouldn't matter.
>>
>> You could also start up new Solrs on the new machines as replicas of the
>> cores you want to move - then once they are active, unload the cores on the
>> old machine, stop the Solr instances and remove the stuff left on the
>> filesystem.
>>
>> - Mark
>>
>> On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com> wrote:
>>
>>> Right now I have an index with four shards on a single EC2 server, each
>>> running on different ports. Now I'd like to migrate three shards
>>> to independent servers.
>>>
>>> What should I do to safely accomplish this process?
>>>
>>> Can I just
>>> 1. shutdown all four solr instances.
>>> 2. copy three shards (indexes) to different servers.
>>> 3. launch 4 solr instances on 4 different servers, each with -zKhost
>>> specified, pointing to the zookeeper servers.
>>>
>>> In my impression, zookeeper remembers which shards are leaders. What I
>>> plan to do above could not elect the three new servers as leaders. If
>> so,
>>> what's the correct way to do it?
>>>
>>> Thanks,
>>> Ming
>>
Re: How to migrate SolrCloud shards to different servers?
Posted by Mingfeng Yang <mf...@wisewindow.com>.
Hi Mark,
When I did testing with SolrCloud, I found the following.
1. I started 4 shards on the same host on port 8983, 8973, 8963, and 8953.
2. Index some data.
3. Shutdown all 4 shards.
4. Started 4 shards again, all pointing to the same data directory and use
the same configuration, except that now we use different ports 8983, 8973,
7633 and 7648.
5. Now Solr has problem to load all cores properly.
Therefore, I had the impression that ZooKeeper may have a memory of which
hosts correspond to which shards. If I change the host info, it may get
confused. I could not find any related documentation or discussion about
this issue.
Thanks,
Ming
On Fri, Jan 25, 2013 at 5:52 PM, Mark Miller <ma...@gmail.com> wrote:
> You could do it that way.
>
> I'm not sure why you are worried about the leaders. That shouldn't matter.
>
> You could also start up new Solrs on the new machines as replicas of the
> cores you want to move - then once they are active, unload the cores on the
> old machine, stop the Solr instances and remove the stuff left on the
> filesystem.
>
> - Mark
>
> On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com> wrote:
>
> > Right now I have an index with four shards on a single EC2 server, each
> > running on different ports. Now I'd like to migrate three shards
> > to independent servers.
> >
> > What should I do to safely accomplish this process?
> >
> > Can I just
> > 1. shutdown all four solr instances.
> > 2. copy three shards (indexes) to different servers.
> > 3. launch 4 solr instances on 4 different servers, each with -zKhost
> > specified, pointing to the zookeeper servers.
> >
> > In my impression, zookeeper remembers which shards are leaders. What I
> > plan to do above could not elect the three new servers as leaders. If
> so,
> > what's the correct way to do it?
> >
> > Thanks,
> > Ming
>
>
Re:Re: How to migrate SolrCloud shards to different servers?
Posted by SuoNayi <su...@163.com>.
As my understand, you mean that we can just add replicas for the already existent shards.
This step is straight.After replicas are active, we can just shutdown the replaced shard nodes
and new leaders will be present automatically. right?
At last we just remove the stuff left on those replaced shard nodes,but why we need unload
the cores on the replaced machines?
At 2013-01-26 09:52:14,"Mark Miller" <ma...@gmail.com> wrote:
>You could do it that way.
>
>I'm not sure why you are worried about the leaders. That shouldn't matter.
>
>You could also start up new Solrs on the new machines as replicas of the cores you want to move - then once they are active, unload the cores on the old machine, stop the Solr instances and remove the stuff left on the filesystem.
>
>- Mark
>
>On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com> wrote:
>
>> Right now I have an index with four shards on a single EC2 server, each
>> running on different ports. Now I'd like to migrate three shards
>> to independent servers.
>>
>> What should I do to safely accomplish this process?
>>
>> Can I just
>> 1. shutdown all four solr instances.
>> 2. copy three shards (indexes) to different servers.
>> 3. launch 4 solr instances on 4 different servers, each with -zKhost
>> specified, pointing to the zookeeper servers.
>>
>> In my impression, zookeeper remembers which shards are leaders. What I
>> plan to do above could not elect the three new servers as leaders. If so,
>> what's the correct way to do it?
>>
>> Thanks,
>> Ming
>
Re: How to migrate SolrCloud shards to different servers?
Posted by Mark Miller <ma...@gmail.com>.
You could do it that way.
I'm not sure why you are worried about the leaders. That shouldn't matter.
You could also start up new Solrs on the new machines as replicas of the cores you want to move - then once they are active, unload the cores on the old machine, stop the Solr instances and remove the stuff left on the filesystem.
- Mark
On Jan 25, 2013, at 7:42 PM, Mingfeng Yang <mf...@wisewindow.com> wrote:
> Right now I have an index with four shards on a single EC2 server, each
> running on different ports. Now I'd like to migrate three shards
> to independent servers.
>
> What should I do to safely accomplish this process?
>
> Can I just
> 1. shutdown all four solr instances.
> 2. copy three shards (indexes) to different servers.
> 3. launch 4 solr instances on 4 different servers, each with -zKhost
> specified, pointing to the zookeeper servers.
>
> In my impression, zookeeper remembers which shards are leaders. What I
> plan to do above could not elect the three new servers as leaders. If so,
> what's the correct way to do it?
>
> Thanks,
> Ming