You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Elaine Cario <et...@gmail.com> on 2017/12/18 17:29:48 UTC
CDCR configuration in solrconfig
We've recently been exploring options for disaster recovery, and took a
look at CDCR for our SolrCloud(s). It seems to meet our needs, but we've
stumbled into a couple of issues with configuration.
The first issue is that currently CDCR is configured as a request handler
in solrconfig, but because we will use the same SolrConfig for collections
in different environments (e.g. development, qa, production), the config
will not always be deployed in an environment that has CDCR. As a last
resort, we are thinking we can drop back to an old-school xml include, and
configure different includes for different environments. This isn't
particularly elegant, but workable. Wondering if anyone has done it some
other way?
The 2nd issue I haven't found a work-around for is the collection name
mapping within the cdcr request handler configuration. For some of our
applications, we "share" the same Solr config with many collections. When
deploying, we just "upconfig" to ZK, and either create a new collection
against that same config (config name != collection name). I'm not sure
with the collection name "baked into" the config how I would manage that,
except to switch to using a dedicated config for each collection.
SOLR-8389 looks like it might solve some of these issues, or at least make
them easier to manage. Is this on the roadmap at all?
Any ideas would be appreciated. Thanks!
Re: CDCR configuration in solrconfig
Posted by Elaine Cario <et...@gmail.com>.
Thanks everyone for your suggestions - I will definitely take a look at the
Config API. We are building more automation into our deployment processes,
and I think we could fit API calls into that.
On Mon, Dec 18, 2017 at 4:16 PM, Webster Homer <we...@sial.com>
wrote:
> We also have the same configurations used in different environments. We
> upload the configset to zookeeper and use the Config API to overlay
> environment specific settings in the solrconfig.xml. We have avoided having
> collections share the same configsets, basically for this reason.
>
> If CDCR supported aliases (SOLR-10679) this would be even easier.
>
> So I suggest using the config API to configure CDCR in each of your
> environments.
>
> On Mon, Dec 18, 2017 at 1:12 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
> > CDCR doesn't do this yet but WDYT about an option where the
> > target collection was _assumed_ to be the same as the source?
> >
> > You're right, SOLR-8389 (and associated) should address this
> > but I don't know what the progress is on that. Seems like
> > a reasonable default in any case.
> >
> > Erick
> >
> > On Mon, Dec 18, 2017 at 9:29 AM, Elaine Cario <et...@gmail.com> wrote:
> > > We've recently been exploring options for disaster recovery, and took a
> > > look at CDCR for our SolrCloud(s). It seems to meet our needs, but
> we've
> > > stumbled into a couple of issues with configuration.
> > >
> > > The first issue is that currently CDCR is configured as a request
> handler
> > > in solrconfig, but because we will use the same SolrConfig for
> > collections
> > > in different environments (e.g. development, qa, production), the
> config
> > > will not always be deployed in an environment that has CDCR. As a last
> > > resort, we are thinking we can drop back to an old-school xml include,
> > and
> > > configure different includes for different environments. This isn't
> > > particularly elegant, but workable. Wondering if anyone has done it
> some
> > > other way?
> > >
> > > The 2nd issue I haven't found a work-around for is the collection name
> > > mapping within the cdcr request handler configuration. For some of our
> > > applications, we "share" the same Solr config with many collections.
> > When
> > > deploying, we just "upconfig" to ZK, and either create a new collection
> > > against that same config (config name != collection name). I'm not
> sure
> > > with the collection name "baked into" the config how I would manage
> that,
> > > except to switch to using a dedicated config for each collection.
> > >
> > > SOLR-8389 looks like it might solve some of these issues, or at least
> > make
> > > them easier to manage. Is this on the roadmap at all?
> > >
> > > Any ideas would be appreciated. Thanks!
> >
>
> --
>
>
> This message and any attachment are confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> you must not copy this message or attachment or disclose the contents to
> any other person. If you have received this transmission in error, please
> notify the sender immediately and delete the message and any attachment
> from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
>
> Click http://www.emdgroup.com/disclaimer to access the German, French,
> Spanish and Portuguese versions of this disclaimer.
>
Re: CDCR configuration in solrconfig
Posted by Webster Homer <we...@sial.com>.
We also have the same configurations used in different environments. We
upload the configset to zookeeper and use the Config API to overlay
environment specific settings in the solrconfig.xml. We have avoided having
collections share the same configsets, basically for this reason.
If CDCR supported aliases (SOLR-10679) this would be even easier.
So I suggest using the config API to configure CDCR in each of your
environments.
On Mon, Dec 18, 2017 at 1:12 PM, Erick Erickson <er...@gmail.com>
wrote:
> CDCR doesn't do this yet but WDYT about an option where the
> target collection was _assumed_ to be the same as the source?
>
> You're right, SOLR-8389 (and associated) should address this
> but I don't know what the progress is on that. Seems like
> a reasonable default in any case.
>
> Erick
>
> On Mon, Dec 18, 2017 at 9:29 AM, Elaine Cario <et...@gmail.com> wrote:
> > We've recently been exploring options for disaster recovery, and took a
> > look at CDCR for our SolrCloud(s). It seems to meet our needs, but we've
> > stumbled into a couple of issues with configuration.
> >
> > The first issue is that currently CDCR is configured as a request handler
> > in solrconfig, but because we will use the same SolrConfig for
> collections
> > in different environments (e.g. development, qa, production), the config
> > will not always be deployed in an environment that has CDCR. As a last
> > resort, we are thinking we can drop back to an old-school xml include,
> and
> > configure different includes for different environments. This isn't
> > particularly elegant, but workable. Wondering if anyone has done it some
> > other way?
> >
> > The 2nd issue I haven't found a work-around for is the collection name
> > mapping within the cdcr request handler configuration. For some of our
> > applications, we "share" the same Solr config with many collections.
> When
> > deploying, we just "upconfig" to ZK, and either create a new collection
> > against that same config (config name != collection name). I'm not sure
> > with the collection name "baked into" the config how I would manage that,
> > except to switch to using a dedicated config for each collection.
> >
> > SOLR-8389 looks like it might solve some of these issues, or at least
> make
> > them easier to manage. Is this on the roadmap at all?
> >
> > Any ideas would be appreciated. Thanks!
>
--
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.
Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Re: CDCR configuration in solrconfig
Posted by Erick Erickson <er...@gmail.com>.
CDCR doesn't do this yet but WDYT about an option where the
target collection was _assumed_ to be the same as the source?
You're right, SOLR-8389 (and associated) should address this
but I don't know what the progress is on that. Seems like
a reasonable default in any case.
Erick
On Mon, Dec 18, 2017 at 9:29 AM, Elaine Cario <et...@gmail.com> wrote:
> We've recently been exploring options for disaster recovery, and took a
> look at CDCR for our SolrCloud(s). It seems to meet our needs, but we've
> stumbled into a couple of issues with configuration.
>
> The first issue is that currently CDCR is configured as a request handler
> in solrconfig, but because we will use the same SolrConfig for collections
> in different environments (e.g. development, qa, production), the config
> will not always be deployed in an environment that has CDCR. As a last
> resort, we are thinking we can drop back to an old-school xml include, and
> configure different includes for different environments. This isn't
> particularly elegant, but workable. Wondering if anyone has done it some
> other way?
>
> The 2nd issue I haven't found a work-around for is the collection name
> mapping within the cdcr request handler configuration. For some of our
> applications, we "share" the same Solr config with many collections. When
> deploying, we just "upconfig" to ZK, and either create a new collection
> against that same config (config name != collection name). I'm not sure
> with the collection name "baked into" the config how I would manage that,
> except to switch to using a dedicated config for each collection.
>
> SOLR-8389 looks like it might solve some of these issues, or at least make
> them easier to manage. Is this on the roadmap at all?
>
> Any ideas would be appreciated. Thanks!