SolrCloud copy the index to another cluster.

[heaven <ah...@gmail.com>]

Hello,

We do have a running SolrCloud cluster, a simple set up of 4 nodes — 2
shards and 2 replicas and ≈ 140GB index. And now we have to move to another
server and need to somehow copy existing index without downtime (if
applicable).

New config is exactly the same, same 4 nodes, same collections and their own
zookeeper.

What options do we have?

What I was thinking about is to add 2 nodes (from the new cluster, those
that are supposed to be shards) as replicas for the existing old cluster and
when the replication is done simply switch the app to use those new
replicas. Then reconfigure these replicas and run them as shards with their
own zookeper. So there will be minimal downtime just to restart the new
cluster.

My concerns are:
* Will those new replicas be automatically populated with the index from the
old cluster?
* Will I then be able to disconnect them from the old cluster and run as
primary shards with their own zookeper and then add their own replicas from
the new cluster?

Thank you,
Alex



--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud copy the index to another cluster.

[heaven <ah...@gmail.com>]

Hi, sorry for the delay.

Yes, we thought to simply copy the index over but this sounds risky and time
consuming. Our index is too big to copy it over the internet quickly.

We decided to re-index our data and then switch and re-index again. It's a
pity there's no way to do this like with mysql :)

Best,
Alex



--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759p4145517.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud copy the index to another cluster.

[Michael Della Bitta <mi...@appinions.com>]

So what I'm playing with now is creating a new collection on the target
cluster, turning off the target cluster, wiping the indexes, and manually
just copying the indexes over to the correct directories and starting
again. In the middle, you can run an optimize or use the Lucene index
upgrader tool to bring yourself up to the new version.

Part of this for me is a migration to HDFSDirectory so there's an added
level of complication there.

I would assume that since you only need to preserve reads, you could cut
over once your collections were created on the new cloud?

Michael Della Bitta

Applications Developer

o: +1 646 532 3062

appinions inc.

“The Science of Influence Marketing”

18 East 41st Street

New York, NY 10017

t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions
<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts>
w: appinions.com <http://www.appinions.com/>


On Tue, Jun 24, 2014 at 3:25 PM, heaven <ah...@gmail.com> wrote:

> Zero read would be enough, we can safely stop index updates for a while.
> But
> have some API endpoints, where read downtime is very undesirable.
>
> Best,
> Alex
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759p4143795.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: SolrCloud copy the index to another cluster.

[heaven <ah...@gmail.com>]

Zero read would be enough, we can safely stop index updates for a while. But
have some API endpoints, where read downtime is very undesirable.

Best,
Alex



--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759p4143795.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud copy the index to another cluster.

[Michael Della Bitta <mi...@appinions.com>]

I'm currently playing around with Solr Cloud migration strategies, too. I'm
wondering... when you say "zero downtime," do you mean zero *read*
downtime, or zero downtime altogether?

Michael Della Bitta

Applications Developer

o: +1 646 532 3062

appinions inc.

“The Science of Influence Marketing”

18 East 41st Street

New York, NY 10017

t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions
<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts>
w: appinions.com <http://www.appinions.com/>


On Tue, Jun 24, 2014 at 1:43 PM, heaven <ah...@gmail.com> wrote:

> I've just realized that old and new clusters do use different
> installations,
> configs and lib paths. So the nodes from the new cluster will probably
> simply refuse to start using configs from the old zookeper.
>
> Only if there is a way to run them with their own zookeper and then
> manually
> add as replicas to the old cluster, so old and new clusters keep using
> their
> own zookepers.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759p4143769.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: SolrCloud copy the index to another cluster.

[heaven <ah...@gmail.com>]

I've just realized that old and new clusters do use different installations,
configs and lib paths. So the nodes from the new cluster will probably
simply refuse to start using configs from the old zookeper.

Only if there is a way to run them with their own zookeper and then manually
add as replicas to the old cluster, so old and new clusters keep using their
own zookepers.



--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-copy-the-index-to-another-cluster-tp4143759p4143769.html
Sent from the Solr - User mailing list archive at Nabble.com.