You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Oakley, Craig (NIH/NLM/NCBI) [C]" <cr...@nih.gov.INVALID> on 2020/09/30 18:58:10 UTC
Master/Slave
Based on the thread below (reading "legacy" as meaning "likely to be deprecated in later versions"), we have been working to extract ourselves from Master/Slave replication
Most of our collections need to be in two data centers (a read/write copy in one local data center: the disaster-recovery-site SolrCloud could be read-only). We also need redundancy within each data center for when one host or another is unavailable. We implemented this by having different SolrClouds in the different data centers; with Master/Slave replication pulling data from one of the read/write replicas to each of the Slave replicas in the disaster-recovery-site read-only SolrCloud. Additionally, for some collections, there is a desire to have local read-only replicas remain unchanged for querying during the loading process: for these collections, there is a local read/write loading SolrCloud, a local read-only querying SolrCloud (normally configured for Master/Slave replication from one of the replicas of the loader SolrCloud to both replicas of the query SolrCloud, but with Master/Slave disabled when the load was in progress on the loader SolrCloud, and with Master/Slave resumed after the loaded data passes QA checks).
Based on the thread below, we made an attempt to switch to CDCR. The main reason for wanting to change was that CDCR was said to be the supported mechanism, and the replacement for Master/Slave replication.
After multiple unsuccessful attempts to get CDCR to work, we ended up with reproducible cases of CDCR loosing data in transit. In June, I initiated a thread in this group asking for clarification of how/whether CDCR could be made reliable. This seemed to me to be met with deafening silence until the announcement in July of the release of Solr8.6 and the deprecation of CDCR.
So we are left with the question whether we should expect Master/Slave replication also to be deprecated; and if so, with what is it expected to be replaced (since not with CDCR)? Or is it now sufficiently safe to assume that Master/Slave replication will continue to be supported after all (since the assertion that it would be replaced by CDCR has been discredited)? In either case, are there other suggested implementations of having a read-only SolrCloud receive data from a read/write SolrCloud?
Thanks
-----Original Message-----
From: Shawn Heisey <ap...@elyograg.org>
Sent: Tuesday, May 21, 2019 11:15 AM
To: solr-user@lucene.apache.org
Subject: Re: SolrCloud (7.3) and Legacy replication slaves
On 5/21/2019 8:48 AM, Michael Tracey wrote:
> Is it possible set up an existing SolrCloud cluster as the master for
> legacy replication to a slave server or two? It looks like another option
> is to use Uni-direction CDCR, but not sure what is the best option in this
> case.
You're asking for problems if you try to combine legacy replication with
SolrCloud. The two features are not guaranteed to work together.
CDCR is your best bet. This replicates from one SolrCloud cluster to
another.
Thanks,
Shawn
Re: Master/Slave
Posted by Eric Pugh <ep...@opensourceconnections.com>.
I’ve met folks who’ve actually used the streaming expressions to move data around if you are looking for a “all Solr” approach. If you go down that route, I’d love to hear how it works.
> On Oct 8, 2020, at 7:10 AM, Erick Erickson <er...@gmail.com> wrote:
>
> What Jan said. I wanted to add that the replication API also makes use of it. A little-known fact: you can use the replication API in SolrCloud _without_ configuring anything in solrconfig.xml. You can specify the URL to pull from on the fly in the command….
>
> Best,
> Erick
>
>> On Oct 8, 2020, at 2:54 AM, Jan Høydahl <ja...@cominvent.com> wrote:
>>
>> The API that enables master/slave is the ReplicationHandler, where the follower (slave) pulls index files from leader (master).
>> This same API is used in SolrCloud for the PULL replica type, and also as a fallback for full recovery if transaction log is not enough.
>> So I don’t see it going away anytime soon, even if the non-cloud deployment style is less promoted in the documentation.
>>
>> Jan
>>
>>> 6. okt. 2020 kl. 16:25 skrev Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov.INVALID>:
>>>
>>>> it better not ever be depreciated. it has been the most reliable mechanism for its purpose
>>>
>>> I would like to know whether that is the consensus of Solr developers.
>>>
>>> We had been scrambling to move from Master/Slave to CDCR based on the assertion that CDCR support would last far longer than Master/Slave support.
>>>
>>> Can we now assume safely that this assertion is now completely moot? Can we now assume safely that Master/Slave is likely to be supported for the foreseeable future? Or are we forced to assume that Master/Slave support will evaporate shortly after the now-evaporated CDCR support?
>>>
>>> -----Original Message-----
>>> From: David Hastings <ha...@gmail.com>
>>> Sent: Wednesday, September 30, 2020 3:10 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Master/Slave
>>>
>>>> whether we should expect Master/Slave replication also to be deprecated
>>>
>>> it better not ever be depreciated. it has been the most reliable mechanism
>>> for its purpose, solr cloud isnt going to replace standalone, if it does,
>>> thats when I guess I stop upgrading or move to elastic
>>>
>>> On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
>>> <cr...@nih.gov.invalid> wrote:
>>>
>>>> Based on the thread below (reading "legacy" as meaning "likely to be
>>>> deprecated in later versions"), we have been working to extract ourselves
>>>> from Master/Slave replication
>>>>
>>>> Most of our collections need to be in two data centers (a read/write copy
>>>> in one local data center: the disaster-recovery-site SolrCloud could be
>>>> read-only). We also need redundancy within each data center for when one
>>>> host or another is unavailable. We implemented this by having different
>>>> SolrClouds in the different data centers; with Master/Slave replication
>>>> pulling data from one of the read/write replicas to each of the Slave
>>>> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
>>>> for some collections, there is a desire to have local read-only replicas
>>>> remain unchanged for querying during the loading process: for these
>>>> collections, there is a local read/write loading SolrCloud, a local
>>>> read-only querying SolrCloud (normally configured for Master/Slave
>>>> replication from one of the replicas of the loader SolrCloud to both
>>>> replicas of the query SolrCloud, but with Master/Slave disabled when the
>>>> load was in progress on the loader SolrCloud, and with Master/Slave resumed
>>>> after the loaded data passes QA checks).
>>>>
>>>> Based on the thread below, we made an attempt to switch to CDCR. The main
>>>> reason for wanting to change was that CDCR was said to be the supported
>>>> mechanism, and the replacement for Master/Slave replication.
>>>>
>>>> After multiple unsuccessful attempts to get CDCR to work, we ended up with
>>>> reproducible cases of CDCR loosing data in transit. In June, I initiated a
>>>> thread in this group asking for clarification of how/whether CDCR could be
>>>> made reliable. This seemed to me to be met with deafening silence until the
>>>> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>>>>
>>>> So we are left with the question whether we should expect Master/Slave
>>>> replication also to be deprecated; and if so, with what is it expected to
>>>> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
>>>> that Master/Slave replication will continue to be supported after all
>>>> (since the assertion that it would be replaced by CDCR has been
>>>> discredited)? In either case, are there other suggested implementations of
>>>> having a read-only SolrCloud receive data from a read/write SolrCloud?
>>>>
>>>>
>>>> Thanks
>>>>
>>>> -----Original Message-----
>>>> From: Shawn Heisey <ap...@elyograg.org>
>>>> Sent: Tuesday, May 21, 2019 11:15 AM
>>>> To: solr-user@lucene.apache.org
>>>> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>>>>
>>>> On 5/21/2019 8:48 AM, Michael Tracey wrote:
>>>>> Is it possible set up an existing SolrCloud cluster as the master for
>>>>> legacy replication to a slave server or two? It looks like another
>>>> option
>>>>> is to use Uni-direction CDCR, but not sure what is the best option in
>>>> this
>>>>> case.
>>>>
>>>> You're asking for problems if you try to combine legacy replication with
>>>> SolrCloud. The two features are not guaranteed to work together.
>>>>
>>>> CDCR is your best bet. This replicates from one SolrCloud cluster to
>>>> another.
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>
>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
What Jan said. I wanted to add that the replication API also makes use of it. A little-known fact: you can use the replication API in SolrCloud _without_ configuring anything in solrconfig.xml. You can specify the URL to pull from on the fly in the command….
Best,
Erick
> On Oct 8, 2020, at 2:54 AM, Jan Høydahl <ja...@cominvent.com> wrote:
>
> The API that enables master/slave is the ReplicationHandler, where the follower (slave) pulls index files from leader (master).
> This same API is used in SolrCloud for the PULL replica type, and also as a fallback for full recovery if transaction log is not enough.
> So I don’t see it going away anytime soon, even if the non-cloud deployment style is less promoted in the documentation.
>
> Jan
>
>> 6. okt. 2020 kl. 16:25 skrev Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov.INVALID>:
>>
>>> it better not ever be depreciated. it has been the most reliable mechanism for its purpose
>>
>> I would like to know whether that is the consensus of Solr developers.
>>
>> We had been scrambling to move from Master/Slave to CDCR based on the assertion that CDCR support would last far longer than Master/Slave support.
>>
>> Can we now assume safely that this assertion is now completely moot? Can we now assume safely that Master/Slave is likely to be supported for the foreseeable future? Or are we forced to assume that Master/Slave support will evaporate shortly after the now-evaporated CDCR support?
>>
>> -----Original Message-----
>> From: David Hastings <ha...@gmail.com>
>> Sent: Wednesday, September 30, 2020 3:10 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Master/Slave
>>
>>> whether we should expect Master/Slave replication also to be deprecated
>>
>> it better not ever be depreciated. it has been the most reliable mechanism
>> for its purpose, solr cloud isnt going to replace standalone, if it does,
>> thats when I guess I stop upgrading or move to elastic
>>
>> On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
>> <cr...@nih.gov.invalid> wrote:
>>
>>> Based on the thread below (reading "legacy" as meaning "likely to be
>>> deprecated in later versions"), we have been working to extract ourselves
>>> from Master/Slave replication
>>>
>>> Most of our collections need to be in two data centers (a read/write copy
>>> in one local data center: the disaster-recovery-site SolrCloud could be
>>> read-only). We also need redundancy within each data center for when one
>>> host or another is unavailable. We implemented this by having different
>>> SolrClouds in the different data centers; with Master/Slave replication
>>> pulling data from one of the read/write replicas to each of the Slave
>>> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
>>> for some collections, there is a desire to have local read-only replicas
>>> remain unchanged for querying during the loading process: for these
>>> collections, there is a local read/write loading SolrCloud, a local
>>> read-only querying SolrCloud (normally configured for Master/Slave
>>> replication from one of the replicas of the loader SolrCloud to both
>>> replicas of the query SolrCloud, but with Master/Slave disabled when the
>>> load was in progress on the loader SolrCloud, and with Master/Slave resumed
>>> after the loaded data passes QA checks).
>>>
>>> Based on the thread below, we made an attempt to switch to CDCR. The main
>>> reason for wanting to change was that CDCR was said to be the supported
>>> mechanism, and the replacement for Master/Slave replication.
>>>
>>> After multiple unsuccessful attempts to get CDCR to work, we ended up with
>>> reproducible cases of CDCR loosing data in transit. In June, I initiated a
>>> thread in this group asking for clarification of how/whether CDCR could be
>>> made reliable. This seemed to me to be met with deafening silence until the
>>> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>>>
>>> So we are left with the question whether we should expect Master/Slave
>>> replication also to be deprecated; and if so, with what is it expected to
>>> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
>>> that Master/Slave replication will continue to be supported after all
>>> (since the assertion that it would be replaced by CDCR has been
>>> discredited)? In either case, are there other suggested implementations of
>>> having a read-only SolrCloud receive data from a read/write SolrCloud?
>>>
>>>
>>> Thanks
>>>
>>> -----Original Message-----
>>> From: Shawn Heisey <ap...@elyograg.org>
>>> Sent: Tuesday, May 21, 2019 11:15 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>>>
>>> On 5/21/2019 8:48 AM, Michael Tracey wrote:
>>>> Is it possible set up an existing SolrCloud cluster as the master for
>>>> legacy replication to a slave server or two? It looks like another
>>> option
>>>> is to use Uni-direction CDCR, but not sure what is the best option in
>>> this
>>>> case.
>>>
>>> You're asking for problems if you try to combine legacy replication with
>>> SolrCloud. The two features are not guaranteed to work together.
>>>
>>> CDCR is your best bet. This replicates from one SolrCloud cluster to
>>> another.
>>>
>>> Thanks,
>>> Shawn
>>>
>
Re: Master/Slave
Posted by Jan Høydahl <ja...@cominvent.com>.
The API that enables master/slave is the ReplicationHandler, where the follower (slave) pulls index files from leader (master).
This same API is used in SolrCloud for the PULL replica type, and also as a fallback for full recovery if transaction log is not enough.
So I don’t see it going away anytime soon, even if the non-cloud deployment style is less promoted in the documentation.
Jan
> 6. okt. 2020 kl. 16:25 skrev Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov.INVALID>:
>
>> it better not ever be depreciated. it has been the most reliable mechanism for its purpose
>
> I would like to know whether that is the consensus of Solr developers.
>
> We had been scrambling to move from Master/Slave to CDCR based on the assertion that CDCR support would last far longer than Master/Slave support.
>
> Can we now assume safely that this assertion is now completely moot? Can we now assume safely that Master/Slave is likely to be supported for the foreseeable future? Or are we forced to assume that Master/Slave support will evaporate shortly after the now-evaporated CDCR support?
>
> -----Original Message-----
> From: David Hastings <ha...@gmail.com>
> Sent: Wednesday, September 30, 2020 3:10 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Master/Slave
>
>> whether we should expect Master/Slave replication also to be deprecated
>
> it better not ever be depreciated. it has been the most reliable mechanism
> for its purpose, solr cloud isnt going to replace standalone, if it does,
> thats when I guess I stop upgrading or move to elastic
>
> On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
> <cr...@nih.gov.invalid> wrote:
>
>> Based on the thread below (reading "legacy" as meaning "likely to be
>> deprecated in later versions"), we have been working to extract ourselves
>> from Master/Slave replication
>>
>> Most of our collections need to be in two data centers (a read/write copy
>> in one local data center: the disaster-recovery-site SolrCloud could be
>> read-only). We also need redundancy within each data center for when one
>> host or another is unavailable. We implemented this by having different
>> SolrClouds in the different data centers; with Master/Slave replication
>> pulling data from one of the read/write replicas to each of the Slave
>> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
>> for some collections, there is a desire to have local read-only replicas
>> remain unchanged for querying during the loading process: for these
>> collections, there is a local read/write loading SolrCloud, a local
>> read-only querying SolrCloud (normally configured for Master/Slave
>> replication from one of the replicas of the loader SolrCloud to both
>> replicas of the query SolrCloud, but with Master/Slave disabled when the
>> load was in progress on the loader SolrCloud, and with Master/Slave resumed
>> after the loaded data passes QA checks).
>>
>> Based on the thread below, we made an attempt to switch to CDCR. The main
>> reason for wanting to change was that CDCR was said to be the supported
>> mechanism, and the replacement for Master/Slave replication.
>>
>> After multiple unsuccessful attempts to get CDCR to work, we ended up with
>> reproducible cases of CDCR loosing data in transit. In June, I initiated a
>> thread in this group asking for clarification of how/whether CDCR could be
>> made reliable. This seemed to me to be met with deafening silence until the
>> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>>
>> So we are left with the question whether we should expect Master/Slave
>> replication also to be deprecated; and if so, with what is it expected to
>> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
>> that Master/Slave replication will continue to be supported after all
>> (since the assertion that it would be replaced by CDCR has been
>> discredited)? In either case, are there other suggested implementations of
>> having a read-only SolrCloud receive data from a read/write SolrCloud?
>>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Shawn Heisey <ap...@elyograg.org>
>> Sent: Tuesday, May 21, 2019 11:15 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>>
>> On 5/21/2019 8:48 AM, Michael Tracey wrote:
>>> Is it possible set up an existing SolrCloud cluster as the master for
>>> legacy replication to a slave server or two? It looks like another
>> option
>>> is to use Uni-direction CDCR, but not sure what is the best option in
>> this
>>> case.
>>
>> You're asking for problems if you try to combine legacy replication with
>> SolrCloud. The two features are not guaranteed to work together.
>>
>> CDCR is your best bet. This replicates from one SolrCloud cluster to
>> another.
>>
>> Thanks,
>> Shawn
>>
RE: Master/Slave
Posted by "Oakley, Craig (NIH/NLM/NCBI) [C]" <cr...@nih.gov.INVALID>.
> it better not ever be depreciated. it has been the most reliable mechanism for its purpose
I would like to know whether that is the consensus of Solr developers.
We had been scrambling to move from Master/Slave to CDCR based on the assertion that CDCR support would last far longer than Master/Slave support.
Can we now assume safely that this assertion is now completely moot? Can we now assume safely that Master/Slave is likely to be supported for the foreseeable future? Or are we forced to assume that Master/Slave support will evaporate shortly after the now-evaporated CDCR support?
-----Original Message-----
From: David Hastings <ha...@gmail.com>
Sent: Wednesday, September 30, 2020 3:10 PM
To: solr-user@lucene.apache.org
Subject: Re: Master/Slave
>whether we should expect Master/Slave replication also to be deprecated
it better not ever be depreciated. it has been the most reliable mechanism
for its purpose, solr cloud isnt going to replace standalone, if it does,
thats when I guess I stop upgrading or move to elastic
On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
<cr...@nih.gov.invalid> wrote:
> Based on the thread below (reading "legacy" as meaning "likely to be
> deprecated in later versions"), we have been working to extract ourselves
> from Master/Slave replication
>
> Most of our collections need to be in two data centers (a read/write copy
> in one local data center: the disaster-recovery-site SolrCloud could be
> read-only). We also need redundancy within each data center for when one
> host or another is unavailable. We implemented this by having different
> SolrClouds in the different data centers; with Master/Slave replication
> pulling data from one of the read/write replicas to each of the Slave
> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
> for some collections, there is a desire to have local read-only replicas
> remain unchanged for querying during the loading process: for these
> collections, there is a local read/write loading SolrCloud, a local
> read-only querying SolrCloud (normally configured for Master/Slave
> replication from one of the replicas of the loader SolrCloud to both
> replicas of the query SolrCloud, but with Master/Slave disabled when the
> load was in progress on the loader SolrCloud, and with Master/Slave resumed
> after the loaded data passes QA checks).
>
> Based on the thread below, we made an attempt to switch to CDCR. The main
> reason for wanting to change was that CDCR was said to be the supported
> mechanism, and the replacement for Master/Slave replication.
>
> After multiple unsuccessful attempts to get CDCR to work, we ended up with
> reproducible cases of CDCR loosing data in transit. In June, I initiated a
> thread in this group asking for clarification of how/whether CDCR could be
> made reliable. This seemed to me to be met with deafening silence until the
> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>
> So we are left with the question whether we should expect Master/Slave
> replication also to be deprecated; and if so, with what is it expected to
> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
> that Master/Slave replication will continue to be supported after all
> (since the assertion that it would be replaced by CDCR has been
> discredited)? In either case, are there other suggested implementations of
> having a read-only SolrCloud receive data from a read/write SolrCloud?
>
>
> Thanks
>
> -----Original Message-----
> From: Shawn Heisey <ap...@elyograg.org>
> Sent: Tuesday, May 21, 2019 11:15 AM
> To: solr-user@lucene.apache.org
> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>
> On 5/21/2019 8:48 AM, Michael Tracey wrote:
> > Is it possible set up an existing SolrCloud cluster as the master for
> > legacy replication to a slave server or two? It looks like another
> option
> > is to use Uni-direction CDCR, but not sure what is the best option in
> this
> > case.
>
> You're asking for problems if you try to combine legacy replication with
> SolrCloud. The two features are not guaranteed to work together.
>
> CDCR is your best bet. This replicates from one SolrCloud cluster to
> another.
>
> Thanks,
> Shawn
>
Re: Master/Slave
Posted by Walter Underwood <wu...@wunderwood.org>.
We do this sort of thing outside of Solr. The indexing process includes creating
a feed file with one JSON object per line. The feed files are stored in S3 with
names that are ISO 8601 timestamps. Those files are picked up and loaded into
Solr. Because S3 is cross-region in AWS, those files are also our disaster
recovery method for indexing. And of course, two clusters could be loaded
from the same file.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Sep 30, 2020, at 12:09 PM, David Hastings <ha...@gmail.com> wrote:
>
>> whether we should expect Master/Slave replication also to be deprecated
>
> it better not ever be depreciated. it has been the most reliable mechanism
> for its purpose, solr cloud isnt going to replace standalone, if it does,
> thats when I guess I stop upgrading or move to elastic
>
> On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
> <cr...@nih.gov.invalid> wrote:
>
>> Based on the thread below (reading "legacy" as meaning "likely to be
>> deprecated in later versions"), we have been working to extract ourselves
>> from Master/Slave replication
>>
>> Most of our collections need to be in two data centers (a read/write copy
>> in one local data center: the disaster-recovery-site SolrCloud could be
>> read-only). We also need redundancy within each data center for when one
>> host or another is unavailable. We implemented this by having different
>> SolrClouds in the different data centers; with Master/Slave replication
>> pulling data from one of the read/write replicas to each of the Slave
>> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
>> for some collections, there is a desire to have local read-only replicas
>> remain unchanged for querying during the loading process: for these
>> collections, there is a local read/write loading SolrCloud, a local
>> read-only querying SolrCloud (normally configured for Master/Slave
>> replication from one of the replicas of the loader SolrCloud to both
>> replicas of the query SolrCloud, but with Master/Slave disabled when the
>> load was in progress on the loader SolrCloud, and with Master/Slave resumed
>> after the loaded data passes QA checks).
>>
>> Based on the thread below, we made an attempt to switch to CDCR. The main
>> reason for wanting to change was that CDCR was said to be the supported
>> mechanism, and the replacement for Master/Slave replication.
>>
>> After multiple unsuccessful attempts to get CDCR to work, we ended up with
>> reproducible cases of CDCR loosing data in transit. In June, I initiated a
>> thread in this group asking for clarification of how/whether CDCR could be
>> made reliable. This seemed to me to be met with deafening silence until the
>> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>>
>> So we are left with the question whether we should expect Master/Slave
>> replication also to be deprecated; and if so, with what is it expected to
>> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
>> that Master/Slave replication will continue to be supported after all
>> (since the assertion that it would be replaced by CDCR has been
>> discredited)? In either case, are there other suggested implementations of
>> having a read-only SolrCloud receive data from a read/write SolrCloud?
>>
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Shawn Heisey <ap...@elyograg.org>
>> Sent: Tuesday, May 21, 2019 11:15 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>>
>> On 5/21/2019 8:48 AM, Michael Tracey wrote:
>>> Is it possible set up an existing SolrCloud cluster as the master for
>>> legacy replication to a slave server or two? It looks like another
>> option
>>> is to use Uni-direction CDCR, but not sure what is the best option in
>> this
>>> case.
>>
>> You're asking for problems if you try to combine legacy replication with
>> SolrCloud. The two features are not guaranteed to work together.
>>
>> CDCR is your best bet. This replicates from one SolrCloud cluster to
>> another.
>>
>> Thanks,
>> Shawn
>>
Re: Master/Slave
Posted by David Hastings <ha...@gmail.com>.
>whether we should expect Master/Slave replication also to be deprecated
it better not ever be depreciated. it has been the most reliable mechanism
for its purpose, solr cloud isnt going to replace standalone, if it does,
thats when I guess I stop upgrading or move to elastic
On Wed, Sep 30, 2020 at 2:58 PM Oakley, Craig (NIH/NLM/NCBI) [C]
<cr...@nih.gov.invalid> wrote:
> Based on the thread below (reading "legacy" as meaning "likely to be
> deprecated in later versions"), we have been working to extract ourselves
> from Master/Slave replication
>
> Most of our collections need to be in two data centers (a read/write copy
> in one local data center: the disaster-recovery-site SolrCloud could be
> read-only). We also need redundancy within each data center for when one
> host or another is unavailable. We implemented this by having different
> SolrClouds in the different data centers; with Master/Slave replication
> pulling data from one of the read/write replicas to each of the Slave
> replicas in the disaster-recovery-site read-only SolrCloud. Additionally,
> for some collections, there is a desire to have local read-only replicas
> remain unchanged for querying during the loading process: for these
> collections, there is a local read/write loading SolrCloud, a local
> read-only querying SolrCloud (normally configured for Master/Slave
> replication from one of the replicas of the loader SolrCloud to both
> replicas of the query SolrCloud, but with Master/Slave disabled when the
> load was in progress on the loader SolrCloud, and with Master/Slave resumed
> after the loaded data passes QA checks).
>
> Based on the thread below, we made an attempt to switch to CDCR. The main
> reason for wanting to change was that CDCR was said to be the supported
> mechanism, and the replacement for Master/Slave replication.
>
> After multiple unsuccessful attempts to get CDCR to work, we ended up with
> reproducible cases of CDCR loosing data in transit. In June, I initiated a
> thread in this group asking for clarification of how/whether CDCR could be
> made reliable. This seemed to me to be met with deafening silence until the
> announcement in July of the release of Solr8.6 and the deprecation of CDCR.
>
> So we are left with the question whether we should expect Master/Slave
> replication also to be deprecated; and if so, with what is it expected to
> be replaced (since not with CDCR)? Or is it now sufficiently safe to assume
> that Master/Slave replication will continue to be supported after all
> (since the assertion that it would be replaced by CDCR has been
> discredited)? In either case, are there other suggested implementations of
> having a read-only SolrCloud receive data from a read/write SolrCloud?
>
>
> Thanks
>
> -----Original Message-----
> From: Shawn Heisey <ap...@elyograg.org>
> Sent: Tuesday, May 21, 2019 11:15 AM
> To: solr-user@lucene.apache.org
> Subject: Re: SolrCloud (7.3) and Legacy replication slaves
>
> On 5/21/2019 8:48 AM, Michael Tracey wrote:
> > Is it possible set up an existing SolrCloud cluster as the master for
> > legacy replication to a slave server or two? It looks like another
> option
> > is to use Uni-direction CDCR, but not sure what is the best option in
> this
> > case.
>
> You're asking for problems if you try to combine legacy replication with
> SolrCloud. The two features are not guaranteed to work together.
>
> CDCR is your best bet. This replicates from one SolrCloud cluster to
> another.
>
> Thanks,
> Shawn
>