Solr Backup/Restore

[Anshuman Singh <si...@gmail.com>]

Hi,

I'm using Solr-7.4.0 and I want to export 4TB of data from our current Solr
cluster to a different cluster. The new cluster has twice the number of
nodes than the current cluster and I want data to be distributed among all
the nodes. Is this possible with the Backup/Restore feature considering the
fact that I want to increase the number of shards in the new Collections?

From the official docs:
*"Support for backups when running SolrCloud is provided with
the Collections API
<https://lucene.apache.org/solr/guide/7_4/collections-api.html#collections-api>.
This allows the backups to be generated across multiple shards, and
restored to the same number of shards and replicas as the original
collection."*

I tried to create a backup of one collection using this feature but it is
giving me this error described here
https://issues.apache.org/jira/browse/SOLR-12523.

Can someone guide me on this and is there any other way to do this exercise
which would take less time?

Thanks,
Anshuman

Re: Solr Backup/Restore

[Aroop Ganguly <ar...@icloud.com.INVALID>]

Restore will only create the same number of shards as the original collection had when you took the backup.
If you are on a cluster with enough resources, you can try split shards to the desired numbers later on?
Split Shards has a more efficient implementation in solr 8.x but if u have a mostly vacant sol 7_4 cluster you can consider splitting shards,
 then taking a backup, and then restoring from this new backup with relevant replica placement rules or at least with maxShardsPerNode=1 or close.
Hope this helps.

> On Jul 21, 2020, at 12:31 PM, Anshuman Singh <si...@gmail.com> wrote:
> 
> Hi,
> 
> I'm using Solr-7.4.0 and I want to export 4TB of data from our current Solr
> cluster to a different cluster. The new cluster has twice the number of
> nodes than the current cluster and I want data to be distributed among all
> the nodes. Is this possible with the Backup/Restore feature considering the
> fact that I want to increase the number of shards in the new Collections?
> 
> From the official docs:
> *"Support for backups when running SolrCloud is provided with
> the Collections API
> <https://lucene.apache.org/solr/guide/7_4/collections-api.html#collections-api>.
> This allows the backups to be generated across multiple shards, and
> restored to the same number of shards and replicas as the original
> collection."*
> 
> I tried to create a backup of one collection using this feature but it is
> giving me this error described here
> https://issues.apache.org/jira/browse/SOLR-12523.
> 
> Can someone guide me on this and is there any other way to do this exercise
> which would take less time?
> 
> Thanks,
> Anshuman