Backup to HDFS while running cluster on local disk

[Mike Thomsen <mi...@gmail.com>]

We have SolrCloud running on bare metal but want the nightly snapshots to
be written to HDFS. Can someone give me some help on configuring the
HdfsBackupRepository?

<backup>
    <repository name="hdfs" class=
"org.apache.solr.core.backup.repository.HdfsBackupRepository" default=
"false">
        <str name="location">${solr.hdfs.default.backup.path}</str>
        <str name="solr.hdfs.home">${solr.hdfs.home:}</str>
        <str name="solr.hdfs.confdir">${solr.hdfs.confdir:}</str>
    </repository>
</backup>

Not sure how to proceed on configuring this because the documentation is a
bit sparse on what some of those values mean in this context. The example
looked geared toward someone using HDFS both to store the index and do
backup/restore.

Thanks,

Mike

Re: Backup to HDFS while running cluster on local disk

[Hrishikesh Gadre <ga...@gmail.com>]

Hi Mike,

Thanks for bringing this up. You can certainly backup the index data stored
on local file-system to HDFS.

The HDFS backup repository implementation uses the same configuration
properties as expected by the HDFS directory factory. Here is the
description of the parameters,

   - location (Optional) - This configuration parameter defines the default
   location where the backups can be stored. If this parameter is not
   configured, then you will need to explicitly specify the location parameter
   to your backup and restore commands.
   - solr.hdfs.home (Required) - This configuration parameter defines the
   fully qualified URI for the root path of HDFS. e.g. hdfs://name-node-1/. In
   case the index files are also stored on HDFS, this path refers to the
   directory used to store index files in HDFS e.g. hdfs://name-node-1/solr
   - solr.hdfs.confdir (Optional) - A directory (on local file-system)
   which contains the configuration files for HDFS (e.g. hdfs-site.xml,
   core-site.xml etc.)


I will also update the docs accordingly.

-Hrishikesh


On Tue, Nov 8, 2016 at 3:36 AM, Mike Thomsen <mi...@gmail.com> wrote:

> We have SolrCloud running on bare metal but want the nightly snapshots to
> be written to HDFS. Can someone give me some help on configuring the
> HdfsBackupRepository?
>
> <backup>
>     <repository name="hdfs" class=
> "org.apache.solr.core.backup.repository.HdfsBackupRepository" default=
> "false">
>         <str name="location">${solr.hdfs.default.backup.path}</str>
>         <str name="solr.hdfs.home">${solr.hdfs.home:}</str>
>         <str name="solr.hdfs.confdir">${solr.hdfs.confdir:}</str>
>     </repository>
> </backup>
>
> Not sure how to proceed on configuring this because the documentation is a
> bit sparse on what some of those values mean in this context. The example
> looked geared toward someone using HDFS both to store the index and do
> backup/restore.
>
> Thanks,
>
> Mike
>