You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Sundaram, Dinesh" <Di...@mastercard.com> on 2018/01/02 22:32:07 UTC
Solrcloud with Master/Slave
Hi,
I have spun up single solrcloud node on 2 servers. tried to synch up the data b/w those servers via zookeeper but didn't work well due to out of memory issues, ensemble issues with multiple ports connectivity. So had to move to Master slave replication b/w those 2 solrcloud nodes. I couldn't find any issues so far. Is this advisable? Because I'm wondering that looks like mixing up solrcloud and master/slave replication.
Dinesh Sundaram
MBS Platform Engineering
Mastercard
[cid:image001.png@01D383E7.39FA10D0]<www.mastercard.com>
CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
Re: Solrcloud with Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
bq: I need to use -cloud in the command while starting solr otherwise I'm
not able to see the admin console
That is totally not true. Yes, you need to start SolrCloud to see the
_cloud section_ of the admin UI, but the rest of the admin UI isn't
dependent on SolrCloud _at all_. In master/slave setups you don't need to
see the cloud section of the admin UI anyway. If you can't see the admin UI
without -cloud, then you're doing something wrong. Or it's a bug, but
that's very doubtful IMO.
You really need to back up and consider that there are many installations
out there that use recent Solrs in master/slave configuration (not cloud at
all, no ZooKeeperes, no -cloud option, just bin/solr start) and function
perfectly fine.
This long conversation is an XY problem. You're asking about specific
actions (X) without explaining what problem you're trying to solve (Y).
This latest (I can't see the admin UI without -cloud) is an example.
Pick one or the other and stick with it. Assume that you're not the only
one trying to do what you're doing and others very probably have been
successful before you make too many assumptions. But please don't mix
SolrCloud with master/slave on your own. It may work but you're really not
using Solr well if you try.
Best,
Erick
On Fri, Jan 5, 2018 at 9:12 AM, Sundaram, Dinesh <
Dinesh.Sundaram@mastercard.com> wrote:
> Thanks Shawn and Erick. I guess now we are in same track. So two
> independent solrcloud nodes are allowed to sync up via master/slave method
> without referring any external/embedded zookeepers. I need to use -cloud in
> the command while starting solr otherwise I'm not able to see the admin
> console. That console is really cool for tracking solr activities.
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Friday, January 5, 2018 10:58 AM
> To: solr-user <so...@lucene.apache.org>
> Subject: Re: Solrcloud with Master/Slave
>
> One slight correction. Solr will run perfectly fine with a single
> ZooKeeper.
> The recommendation for 3 is that running with a single ZooKeeper creates a
> single point of failure, i.e. if that node goes down for any reason your
> Solr cluster won't be able to update anything at all. You can still query,
> maybe, for a while.
>
> Two ZooKeepers will also run, but as Shawn says that's essentially totally
> wasting one of them as it doesn't buy you anything and makes your system
> _less_ robust.
>
> FWIW,
> Erick
>
> On Fri, Jan 5, 2018 at 7:57 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 1/4/2018 9:01 AM, Sundaram, Dinesh wrote:
> > > Thanks Shawn for your prompt response. Assume I have solrcloud A
> > > server
> > with 1 node runs on 8983 port and solrcloud B server with 1 node runs
> > on 8983, here I want to synch up the collection between solrcloud A
> > and B using the below replication handler. Is this advisable to use at
> > the solrcloud B ?
> > >
> > > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > > <lst name="slave">
> > > <str name="masterUrl">https://urldefense.proofpoint.com/v2/
> url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr&d=DwIBaQ&c=
> uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jmCF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-
> 3R6gq7ojE0Eam2Eg&m=3jqDjqHZnl2sdIUhYF4uQBQQBHzg6zEshRmcDCPcWvM&s=tEY-
> h4ureY9H6AkD-b9wGLmOfYQ3NJp3Rg37lNBuPgY&e=.
> > core.name}/replication</str>
> > > <str name="pollInterval">00:00:20</str>
> > > </lst>
> > > </requestHandler>
> >
> > One of the things I said in my last reply, at the beginning of a
> > paragraph so it should have been quite prominent, was "you can't mix
> > master-slave replication and SolrCloud." What part of that was not
> clear?
> >
> > You need to be running standalone mode (not cloud) if you want to use
> > master-slave replication.
> >
> > When things are set up correctly, SolrCloud will automatically keep
> > multiple replicas in sync, and copy the index to new replicas when
> > they are created. There is no need to manage it with replication config.
> > For replicating from one SolrCloud cluster to another, there is CDCR
> > as Erick described.
> >
> > Another thing Erick mentioned: What you actually have when you start
> > Solr the way you did is two completely independent SolrCloud clusters,
> > each of which only has one Solr server. Each solr instance is running
> > a zookeeper server embedded within it. There is no redundancy or
> > fault tolerance of any kind.
> >
> > If you want to run a fault-tolerant SolrCloud, you will need three
> > separate servers. The smallest possible setup would have both Solr
> > and ZooKeeper running on two of those servers (as separate processes).
> > The Solr instances would be started with a -z option (or the ZKHOST
> > environment variable) to locate the three ZK servers, and without the
> > -cloud option. The third server, which can be a much smaller system,
> > would only run ZooKeeper. You may also need a load balancer,
> > depending on what software your clients are using.
> >
> > The requirement of three servers comes from ZooKeeper, not Solr. A
> > two-server ZK ensemble is actually *less* reliable than a single
> > server, so it's not recommended. I don't know if they even allow such
> > a setup to work.
> >
> > Thanks,
> > Shawn
> >
> >
> CONFIDENTIALITY NOTICE This e-mail message and any attachments are only
> for the use of the intended recipient and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient, any disclosure, distribution or other
> use of this e-mail message or attachments is prohibited. If you have
> received this e-mail message in error, please delete and notify the sender
> immediately. Thank you.
>
RE: Solrcloud with Master/Slave
Posted by "Sundaram, Dinesh" <Di...@mastercard.com>.
Thanks Shawn and Erick. I guess now we are in same track. So two independent solrcloud nodes are allowed to sync up via master/slave method without referring any external/embedded zookeepers. I need to use -cloud in the command while starting solr otherwise I'm not able to see the admin console. That console is really cool for tracking solr activities.
Dinesh Sundaram
MBS Platform Engineering
Mastercard
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Friday, January 5, 2018 10:58 AM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Solrcloud with Master/Slave
One slight correction. Solr will run perfectly fine with a single ZooKeeper.
The recommendation for 3 is that running with a single ZooKeeper creates a single point of failure, i.e. if that node goes down for any reason your Solr cluster won't be able to update anything at all. You can still query, maybe, for a while.
Two ZooKeepers will also run, but as Shawn says that's essentially totally wasting one of them as it doesn't buy you anything and makes your system _less_ robust.
FWIW,
Erick
On Fri, Jan 5, 2018 at 7:57 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/4/2018 9:01 AM, Sundaram, Dinesh wrote:
> > Thanks Shawn for your prompt response. Assume I have solrcloud A
> > server
> with 1 node runs on 8983 port and solrcloud B server with 1 node runs
> on 8983, here I want to synch up the collection between solrcloud A
> and B using the below replication handler. Is this advisable to use at
> the solrcloud B ?
> >
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="slave">
> > <str name="masterUrl">https://urldefense.proofpoint.com/v2/url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr&d=DwIBaQ&c=uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jmCF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-3R6gq7ojE0Eam2Eg&m=3jqDjqHZnl2sdIUhYF4uQBQQBHzg6zEshRmcDCPcWvM&s=tEY-h4ureY9H6AkD-b9wGLmOfYQ3NJp3Rg37lNBuPgY&e=.
> core.name}/replication</str>
> > <str name="pollInterval">00:00:20</str>
> > </lst>
> > </requestHandler>
>
> One of the things I said in my last reply, at the beginning of a
> paragraph so it should have been quite prominent, was "you can't mix
> master-slave replication and SolrCloud." What part of that was not clear?
>
> You need to be running standalone mode (not cloud) if you want to use
> master-slave replication.
>
> When things are set up correctly, SolrCloud will automatically keep
> multiple replicas in sync, and copy the index to new replicas when
> they are created. There is no need to manage it with replication config.
> For replicating from one SolrCloud cluster to another, there is CDCR
> as Erick described.
>
> Another thing Erick mentioned: What you actually have when you start
> Solr the way you did is two completely independent SolrCloud clusters,
> each of which only has one Solr server. Each solr instance is running
> a zookeeper server embedded within it. There is no redundancy or
> fault tolerance of any kind.
>
> If you want to run a fault-tolerant SolrCloud, you will need three
> separate servers. The smallest possible setup would have both Solr
> and ZooKeeper running on two of those servers (as separate processes).
> The Solr instances would be started with a -z option (or the ZKHOST
> environment variable) to locate the three ZK servers, and without the
> -cloud option. The third server, which can be a much smaller system,
> would only run ZooKeeper. You may also need a load balancer,
> depending on what software your clients are using.
>
> The requirement of three servers comes from ZooKeeper, not Solr. A
> two-server ZK ensemble is actually *less* reliable than a single
> server, so it's not recommended. I don't know if they even allow such
> a setup to work.
>
> Thanks,
> Shawn
>
>
CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
Re: Solrcloud with Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
One slight correction. Solr will run perfectly fine with a single ZooKeeper.
The recommendation for 3 is that running with a single ZooKeeper creates
a single point of failure, i.e. if that node goes down for any reason your
Solr cluster won't be able to update anything at all. You can still query,
maybe, for a while.
Two ZooKeepers will also run, but as Shawn says that's essentially
totally wasting one of them as it doesn't buy you anything and makes
your system _less_ robust.
FWIW,
Erick
On Fri, Jan 5, 2018 at 7:57 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 1/4/2018 9:01 AM, Sundaram, Dinesh wrote:
> > Thanks Shawn for your prompt response. Assume I have solrcloud A server
> with 1 node runs on 8983 port and solrcloud B server with 1 node runs on
> 8983, here I want to synch up the collection between solrcloud A and B
> using the below replication handler. Is this advisable to use at the
> solrcloud B ?
> >
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="slave">
> > <str name="masterUrl">http://solrcloudA:8983/solr/${solr.
> core.name}/replication</str>
> > <str name="pollInterval">00:00:20</str>
> > </lst>
> > </requestHandler>
>
> One of the things I said in my last reply, at the beginning of a
> paragraph so it should have been quite prominent, was "you can't mix
> master-slave replication and SolrCloud." What part of that was not clear?
>
> You need to be running standalone mode (not cloud) if you want to use
> master-slave replication.
>
> When things are set up correctly, SolrCloud will automatically keep
> multiple replicas in sync, and copy the index to new replicas when they
> are created. There is no need to manage it with replication config.
> For replicating from one SolrCloud cluster to another, there is CDCR as
> Erick described.
>
> Another thing Erick mentioned: What you actually have when you start
> Solr the way you did is two completely independent SolrCloud clusters,
> each of which only has one Solr server. Each solr instance is running a
> zookeeper server embedded within it. There is no redundancy or fault
> tolerance of any kind.
>
> If you want to run a fault-tolerant SolrCloud, you will need three
> separate servers. The smallest possible setup would have both Solr and
> ZooKeeper running on two of those servers (as separate processes). The
> Solr instances would be started with a -z option (or the ZKHOST
> environment variable) to locate the three ZK servers, and without the
> -cloud option. The third server, which can be a much smaller system,
> would only run ZooKeeper. You may also need a load balancer, depending
> on what software your clients are using.
>
> The requirement of three servers comes from ZooKeeper, not Solr. A
> two-server ZK ensemble is actually *less* reliable than a single server,
> so it's not recommended. I don't know if they even allow such a setup
> to work.
>
> Thanks,
> Shawn
>
>
Re: Solrcloud with Master/Slave
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/4/2018 9:01 AM, Sundaram, Dinesh wrote:
> Thanks Shawn for your prompt response. Assume I have solrcloud A server with 1 node runs on 8983 port and solrcloud B server with 1 node runs on 8983, here I want to synch up the collection between solrcloud A and B using the below replication handler. Is this advisable to use at the solrcloud B ?
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="slave">
> <str name="masterUrl">http://solrcloudA:8983/solr/${solr.core.name}/replication</str>
> <str name="pollInterval">00:00:20</str>
> </lst>
> </requestHandler>
One of the things I said in my last reply, at the beginning of a
paragraph so it should have been quite prominent, was "you can't mix
master-slave replication and SolrCloud." What part of that was not clear?
You need to be running standalone mode (not cloud) if you want to use
master-slave replication.
When things are set up correctly, SolrCloud will automatically keep
multiple replicas in sync, and copy the index to new replicas when they
are created. There is no need to manage it with replication config.
For replicating from one SolrCloud cluster to another, there is CDCR as
Erick described.
Another thing Erick mentioned: What you actually have when you start
Solr the way you did is two completely independent SolrCloud clusters,
each of which only has one Solr server. Each solr instance is running a
zookeeper server embedded within it. There is no redundancy or fault
tolerance of any kind.
If you want to run a fault-tolerant SolrCloud, you will need three
separate servers. The smallest possible setup would have both Solr and
ZooKeeper running on two of those servers (as separate processes). The
Solr instances would be started with a -z option (or the ZKHOST
environment variable) to locate the three ZK servers, and without the
-cloud option. The third server, which can be a much smaller system,
would only run ZooKeeper. You may also need a load balancer, depending
on what software your clients are using.
The requirement of three servers comes from ZooKeeper, not Solr. A
two-server ZK ensemble is actually *less* reliable than a single server,
so it's not recommended. I don't know if they even allow such a setup
to work.
Thanks,
Shawn
Re: Solrcloud with Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
As I said before, you _are_using ZooKeeper by starting your servers with
the -cloud option. Just leave that off and you won't be and can use
master/slave freely.
Best,
Erick
On Thu, Jan 4, 2018 at 12:21 PM, Sundaram, Dinesh <
Dinesh.Sundaram@mastercard.com> wrote:
> Ok thanks for your valuable reply. I want to see admin console so that I
> can monitor the collection details, that is the reason going to cloud mode.
> But here I need replication without zookeeper so had to choose regular
> master/slave replication. Am I mixing 2 different synchup procedures or
> this also okay?
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, January 4, 2018 2:06 PM
> To: solr-user <so...@lucene.apache.org>
> Subject: Re: Solrcloud with Master/Slave
>
> Yes you do use ZooKeeper. Starting Solr with the -cloud option but
> _without_ ZK_HOST defined (or the -z parameter) starts an internal
> ZooKeeper on port
> 9983 (by default). This is evidenced by the fact that the admin UI has a
> "cloud" link along the left. In essence you have two separate clusters,
> each cluster just happens to exist on the same machine.
>
> Why bother with SolrCloud? Just configure old-style master/slave.
> SolrCloud is buying you nothing and running internal ZooKeepers is
> consuming resources for no good purpose.
>
> SolrCloud would help you if you set up a proper cluster with ZooKeeper and
> just had both of your nodes in the same cluster, one with replicas. That
> buys you HA/DR, NRT on both leader and follower etc.
>
> Up to you of course, but it's really hard to see what the purpose of
> running the way you are is.
>
> Best,
> Erick
>
> On Thu, Jan 4, 2018 at 11:38 AM, Sundaram, Dinesh <
> Dinesh.Sundaram@mastercard.com> wrote:
>
> > I want to keep both collections in sync always. This is really working
> > fine without any issue so far. My problem is pretty straight forward.
> >
> > I'm starting two solr instances on two servers using the below
> > command. I believe this command is for solrcloud mode. If so then I
> > have that shared replication handler config also in in my
> > _default/solrconfig.xml on one instance so that the slave instance
> > will synch with master. I don’t use zookeeper at all. Just replication
> > handler setting in solrconfig.xml. is this good for longtime? If not
> please help me understand the issues.
> >
> > bin/solr start -cloud -p 8983 -noprompt
> >
> >
> >
> > Dinesh Sundaram
> > MBS Platform Engineering
> >
> > Mastercard
> >
> >
> >
> > -----Original Message-----
> > From: Erick Erickson [mailto:erickerickson@gmail.com]
> > Sent: Thursday, January 4, 2018 10:10 AM
> > To: solr-user <so...@lucene.apache.org>
> > Subject: Re: Solrcloud with Master/Slave
> >
> > Whoa. I don't think you should be doing this at all. This really
> > appears to be an XY problem. You're asking "how to do X" without
> > telling us what the problem you're trying to solve is (the Y). _Why_
> > do you want to set things up this way? A one-time synchronization or
> > to keep both collections in sync?
> >
> >
> > Cross Data Center Replication (CDCR) is designed to keep two separate
> > collections in sync on an ongoing basis.
> >
> > If this is a one-time deal, you can manually issue a replication API
> > "fetchindex" command. What I'd do in that case is set up your
> > collection B with each shard having exactly one replica (i.e. a leader
> > and no followers). Do the fetch and verify that your new collection is
> > as you want it then ADDREPLICA to build out your redundancy.
> >
> > Best,
> > Erick
> >
> > On Thu, Jan 4, 2018 at 8:01 AM, Sundaram, Dinesh <
> > Dinesh.Sundaram@mastercard.com> wrote:
> > > Thanks Shawn for your prompt response. Assume I have solrcloud A
> > > server
> > with 1 node runs on 8983 port and solrcloud B server with 1 node runs
> > on 8983, here I want to synch up the collection between solrcloud A
> > and B using the below replication handler. Is this advisable to use at
> > the solrcloud B ?
> > >
> > > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > > <lst name="slave">
> > > <str name="masterUrl">https://urldefense.proofpoint.com/v2/
> > url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr.core.
> > name-257D_replication&d=DwIFaQ&c=uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jm
> > CF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-3R6gq7ojE0Eam2Eg&m=
> > qBCZxvmkOHW9jt8JM8dVSQJuulIJp3Xk2hXvC5bL7DM&s=xGP-
> > 8z2aGBFGrtjIbBMFB6f2cfE4bukyOctAVK_HkyI&e=</str>
> > > <str name="pollInterval">00:00:20</str>
> > > </lst>
> > > </requestHandler>
> > >
> > >
> > >
> > > Dinesh Sundaram
> > > MBS Platform Engineering
> > >
> > > Mastercard
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Shawn Heisey [mailto:apache@elyograg.org]
> > > Sent: Tuesday, January 2, 2018 5:33 PM
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Solrcloud with Master/Slave
> > >
> > > On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
> > >> I have spun up single solrcloud node on 2 servers.
> > >
> > > This makes no sense. If you have two servers, then you probably
> > > have
> > more than a single node.
> > >
> > >> tried to synch up the data b/w those servers via zookeeper
> > >
> > > This is not done with zookeeper. SolrCloud should handle it
> > automatically. SolrCloud uses the zookeeper database to *coordinate*
> > keeping machines in sync, but it's Solr that does the work, not
> zookeeper.
> > >
> > > This makes even less sense when taken in context with the previous
> > sentence. If you only have a single node, then you can't possibly
> > sync between them.
> > >
> > >> but didn’t work well due to out of memory issues, ensemble issues
> > >> with multiple ports connectivity. So had to move to Master slave
> > >> replication b/w those 2 solrcloud nodes. I couldn’t find any issues
> > >> so far. Is this advisable? Because I’m wondering that looks like
> > >> mixing up solrcloud and master/slave replication.
> > >
> > > If you're getting OOME problems, then whatever program threw the
> > > OOME
> > most likely needs more heap. Or you need to take steps to reduce the
> > amount of heap that's required. Note that this second option might
> > not actually be possible ... increasing the heap is probably the only
> > option you have. Since version 5.0, Solr has shipped with the default
> > heap set to 512MB, which is extremely small. Most users need to
> increase it.
> > >
> > > You can't mix master-slave replication and SolrCloud. SolrCloud
> > > takes
> > over the replication feature for its own purposes. Trying to mix
> > these is going to cause you problems. You may not run into the
> > problems immediately, but it is likely that you would run into a
> > problem eventually. Data loss would be possible.
> > >
> > > The latest versions of Solr have new SolrCloud replication types
> > > that
> > closely mimic the old master-slave replication.
> > >
> > > Perhaps you should start over and describe what you've actually seen
> > > --
> > exactly what you've done and configured, and how the results differed
> > from your expectations. Precise commands entered will be helpful.
> > >
> > > Thanks,
> > > Shawn
> > >
> > >
> > > CONFIDENTIALITY NOTICE This e-mail message and any attachments are
> > > only
> > for the use of the intended recipient and may contain information that
> > is privileged, confidential or exempt from disclosure under applicable
> > law. If you are not the intended recipient, any disclosure,
> > distribution or other use of this e-mail message or attachments is
> > prohibited. If you have received this e-mail message in error, please
> > delete and notify the sender immediately. Thank you.
> >
> >
> CONFIDENTIALITY NOTICE This e-mail message and any attachments are only
> for the use of the intended recipient and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient, any disclosure, distribution or other
> use of this e-mail message or attachments is prohibited. If you have
> received this e-mail message in error, please delete and notify the sender
> immediately. Thank you.
>
RE: Solrcloud with Master/Slave
Posted by "Sundaram, Dinesh" <Di...@mastercard.com>.
Ok thanks for your valuable reply. I want to see admin console so that I can monitor the collection details, that is the reason going to cloud mode. But here I need replication without zookeeper so had to choose regular master/slave replication. Am I mixing 2 different synchup procedures or this also okay?
Dinesh Sundaram
MBS Platform Engineering
Mastercard
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, January 4, 2018 2:06 PM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Solrcloud with Master/Slave
Yes you do use ZooKeeper. Starting Solr with the -cloud option but _without_ ZK_HOST defined (or the -z parameter) starts an internal ZooKeeper on port
9983 (by default). This is evidenced by the fact that the admin UI has a "cloud" link along the left. In essence you have two separate clusters, each cluster just happens to exist on the same machine.
Why bother with SolrCloud? Just configure old-style master/slave.
SolrCloud is buying you nothing and running internal ZooKeepers is consuming resources for no good purpose.
SolrCloud would help you if you set up a proper cluster with ZooKeeper and just had both of your nodes in the same cluster, one with replicas. That buys you HA/DR, NRT on both leader and follower etc.
Up to you of course, but it's really hard to see what the purpose of running the way you are is.
Best,
Erick
On Thu, Jan 4, 2018 at 11:38 AM, Sundaram, Dinesh < Dinesh.Sundaram@mastercard.com> wrote:
> I want to keep both collections in sync always. This is really working
> fine without any issue so far. My problem is pretty straight forward.
>
> I'm starting two solr instances on two servers using the below
> command. I believe this command is for solrcloud mode. If so then I
> have that shared replication handler config also in in my
> _default/solrconfig.xml on one instance so that the slave instance
> will synch with master. I don’t use zookeeper at all. Just replication
> handler setting in solrconfig.xml. is this good for longtime? If not please help me understand the issues.
>
> bin/solr start -cloud -p 8983 -noprompt
>
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, January 4, 2018 10:10 AM
> To: solr-user <so...@lucene.apache.org>
> Subject: Re: Solrcloud with Master/Slave
>
> Whoa. I don't think you should be doing this at all. This really
> appears to be an XY problem. You're asking "how to do X" without
> telling us what the problem you're trying to solve is (the Y). _Why_
> do you want to set things up this way? A one-time synchronization or
> to keep both collections in sync?
>
>
> Cross Data Center Replication (CDCR) is designed to keep two separate
> collections in sync on an ongoing basis.
>
> If this is a one-time deal, you can manually issue a replication API
> "fetchindex" command. What I'd do in that case is set up your
> collection B with each shard having exactly one replica (i.e. a leader
> and no followers). Do the fetch and verify that your new collection is
> as you want it then ADDREPLICA to build out your redundancy.
>
> Best,
> Erick
>
> On Thu, Jan 4, 2018 at 8:01 AM, Sundaram, Dinesh <
> Dinesh.Sundaram@mastercard.com> wrote:
> > Thanks Shawn for your prompt response. Assume I have solrcloud A
> > server
> with 1 node runs on 8983 port and solrcloud B server with 1 node runs
> on 8983, here I want to synch up the collection between solrcloud A
> and B using the below replication handler. Is this advisable to use at
> the solrcloud B ?
> >
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="slave">
> > <str name="masterUrl">https://urldefense.proofpoint.com/v2/
> url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr.core.
> name-257D_replication&d=DwIFaQ&c=uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jm
> CF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-3R6gq7ojE0Eam2Eg&m=
> qBCZxvmkOHW9jt8JM8dVSQJuulIJp3Xk2hXvC5bL7DM&s=xGP-
> 8z2aGBFGrtjIbBMFB6f2cfE4bukyOctAVK_HkyI&e=</str>
> > <str name="pollInterval">00:00:20</str>
> > </lst>
> > </requestHandler>
> >
> >
> >
> > Dinesh Sundaram
> > MBS Platform Engineering
> >
> > Mastercard
> >
> >
> >
> > -----Original Message-----
> > From: Shawn Heisey [mailto:apache@elyograg.org]
> > Sent: Tuesday, January 2, 2018 5:33 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Solrcloud with Master/Slave
> >
> > On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
> >> I have spun up single solrcloud node on 2 servers.
> >
> > This makes no sense. If you have two servers, then you probably
> > have
> more than a single node.
> >
> >> tried to synch up the data b/w those servers via zookeeper
> >
> > This is not done with zookeeper. SolrCloud should handle it
> automatically. SolrCloud uses the zookeeper database to *coordinate*
> keeping machines in sync, but it's Solr that does the work, not zookeeper.
> >
> > This makes even less sense when taken in context with the previous
> sentence. If you only have a single node, then you can't possibly
> sync between them.
> >
> >> but didn’t work well due to out of memory issues, ensemble issues
> >> with multiple ports connectivity. So had to move to Master slave
> >> replication b/w those 2 solrcloud nodes. I couldn’t find any issues
> >> so far. Is this advisable? Because I’m wondering that looks like
> >> mixing up solrcloud and master/slave replication.
> >
> > If you're getting OOME problems, then whatever program threw the
> > OOME
> most likely needs more heap. Or you need to take steps to reduce the
> amount of heap that's required. Note that this second option might
> not actually be possible ... increasing the heap is probably the only
> option you have. Since version 5.0, Solr has shipped with the default
> heap set to 512MB, which is extremely small. Most users need to increase it.
> >
> > You can't mix master-slave replication and SolrCloud. SolrCloud
> > takes
> over the replication feature for its own purposes. Trying to mix
> these is going to cause you problems. You may not run into the
> problems immediately, but it is likely that you would run into a
> problem eventually. Data loss would be possible.
> >
> > The latest versions of Solr have new SolrCloud replication types
> > that
> closely mimic the old master-slave replication.
> >
> > Perhaps you should start over and describe what you've actually seen
> > --
> exactly what you've done and configured, and how the results differed
> from your expectations. Precise commands entered will be helpful.
> >
> > Thanks,
> > Shawn
> >
> >
> > CONFIDENTIALITY NOTICE This e-mail message and any attachments are
> > only
> for the use of the intended recipient and may contain information that
> is privileged, confidential or exempt from disclosure under applicable
> law. If you are not the intended recipient, any disclosure,
> distribution or other use of this e-mail message or attachments is
> prohibited. If you have received this e-mail message in error, please
> delete and notify the sender immediately. Thank you.
>
>
CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
Re: Solrcloud with Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
Yes you do use ZooKeeper. Starting Solr with the -cloud option but _without_
ZK_HOST defined (or the -z parameter) starts an internal ZooKeeper on port
9983 (by default). This is evidenced by the fact that the admin UI has a
"cloud" link along the left. In essence you have two separate clusters,
each cluster
just happens to exist on the same machine.
Why bother with SolrCloud? Just configure old-style master/slave.
SolrCloud is buying you nothing and running internal ZooKeepers is consuming
resources for no good purpose.
SolrCloud would help you if you set up a proper cluster with ZooKeeper
and just had both of your nodes in the same cluster, one with replicas. That
buys you HA/DR, NRT on both leader and follower etc.
Up to you of course, but it's really hard to see what the purpose of
running the
way you are is.
Best,
Erick
On Thu, Jan 4, 2018 at 11:38 AM, Sundaram, Dinesh <
Dinesh.Sundaram@mastercard.com> wrote:
> I want to keep both collections in sync always. This is really working
> fine without any issue so far. My problem is pretty straight forward.
>
> I'm starting two solr instances on two servers using the below command. I
> believe this command is for solrcloud mode. If so then I have that shared
> replication handler config also in in my _default/solrconfig.xml on one
> instance so that the slave instance will synch with master. I don’t use
> zookeeper at all. Just replication handler setting in solrconfig.xml. is
> this good for longtime? If not please help me understand the issues.
>
> bin/solr start -cloud -p 8983 -noprompt
>
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, January 4, 2018 10:10 AM
> To: solr-user <so...@lucene.apache.org>
> Subject: Re: Solrcloud with Master/Slave
>
> Whoa. I don't think you should be doing this at all. This really appears
> to be an XY problem. You're asking "how to do X" without telling us what
> the problem you're trying to solve is (the Y). _Why_ do you want to set
> things up this way? A one-time synchronization or to keep both collections
> in sync?
>
>
> Cross Data Center Replication (CDCR) is designed to keep two separate
> collections in sync on an ongoing basis.
>
> If this is a one-time deal, you can manually issue a replication API
> "fetchindex" command. What I'd do in that case is set up your collection B
> with each shard having exactly one replica (i.e. a leader and no
> followers). Do the fetch and verify that your new collection is as you want
> it then ADDREPLICA to build out your redundancy.
>
> Best,
> Erick
>
> On Thu, Jan 4, 2018 at 8:01 AM, Sundaram, Dinesh <
> Dinesh.Sundaram@mastercard.com> wrote:
> > Thanks Shawn for your prompt response. Assume I have solrcloud A server
> with 1 node runs on 8983 port and solrcloud B server with 1 node runs on
> 8983, here I want to synch up the collection between solrcloud A and B
> using the below replication handler. Is this advisable to use at the
> solrcloud B ?
> >
> > <requestHandler name="/replication" class="solr.ReplicationHandler" >
> > <lst name="slave">
> > <str name="masterUrl">https://urldefense.proofpoint.com/v2/
> url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr.core.
> name-257D_replication&d=DwIFaQ&c=uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jm
> CF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-3R6gq7ojE0Eam2Eg&m=
> qBCZxvmkOHW9jt8JM8dVSQJuulIJp3Xk2hXvC5bL7DM&s=xGP-
> 8z2aGBFGrtjIbBMFB6f2cfE4bukyOctAVK_HkyI&e=</str>
> > <str name="pollInterval">00:00:20</str>
> > </lst>
> > </requestHandler>
> >
> >
> >
> > Dinesh Sundaram
> > MBS Platform Engineering
> >
> > Mastercard
> >
> >
> >
> > -----Original Message-----
> > From: Shawn Heisey [mailto:apache@elyograg.org]
> > Sent: Tuesday, January 2, 2018 5:33 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Solrcloud with Master/Slave
> >
> > On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
> >> I have spun up single solrcloud node on 2 servers.
> >
> > This makes no sense. If you have two servers, then you probably have
> more than a single node.
> >
> >> tried to synch up the data b/w those servers via zookeeper
> >
> > This is not done with zookeeper. SolrCloud should handle it
> automatically. SolrCloud uses the zookeeper database to *coordinate*
> keeping machines in sync, but it's Solr that does the work, not zookeeper.
> >
> > This makes even less sense when taken in context with the previous
> sentence. If you only have a single node, then you can't possibly sync
> between them.
> >
> >> but didn’t work well due to out of memory issues, ensemble issues
> >> with multiple ports connectivity. So had to move to Master slave
> >> replication b/w those 2 solrcloud nodes. I couldn’t find any issues
> >> so far. Is this advisable? Because I’m wondering that looks like
> >> mixing up solrcloud and master/slave replication.
> >
> > If you're getting OOME problems, then whatever program threw the OOME
> most likely needs more heap. Or you need to take steps to reduce the
> amount of heap that's required. Note that this second option might not
> actually be possible ... increasing the heap is probably the only option
> you have. Since version 5.0, Solr has shipped with the default heap set to
> 512MB, which is extremely small. Most users need to increase it.
> >
> > You can't mix master-slave replication and SolrCloud. SolrCloud takes
> over the replication feature for its own purposes. Trying to mix these is
> going to cause you problems. You may not run into the problems
> immediately, but it is likely that you would run into a problem
> eventually. Data loss would be possible.
> >
> > The latest versions of Solr have new SolrCloud replication types that
> closely mimic the old master-slave replication.
> >
> > Perhaps you should start over and describe what you've actually seen --
> exactly what you've done and configured, and how the results differed from
> your expectations. Precise commands entered will be helpful.
> >
> > Thanks,
> > Shawn
> >
> >
> > CONFIDENTIALITY NOTICE This e-mail message and any attachments are only
> for the use of the intended recipient and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient, any disclosure, distribution or other
> use of this e-mail message or attachments is prohibited. If you have
> received this e-mail message in error, please delete and notify the sender
> immediately. Thank you.
>
>
RE: Solrcloud with Master/Slave
Posted by "Sundaram, Dinesh" <Di...@mastercard.com>.
I want to keep both collections in sync always. This is really working fine without any issue so far. My problem is pretty straight forward.
I'm starting two solr instances on two servers using the below command. I believe this command is for solrcloud mode. If so then I have that shared replication handler config also in in my _default/solrconfig.xml on one instance so that the slave instance will synch with master. I don’t use zookeeper at all. Just replication handler setting in solrconfig.xml. is this good for longtime? If not please help me understand the issues.
bin/solr start -cloud -p 8983 -noprompt
Dinesh Sundaram
MBS Platform Engineering
Mastercard
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, January 4, 2018 10:10 AM
To: solr-user <so...@lucene.apache.org>
Subject: Re: Solrcloud with Master/Slave
Whoa. I don't think you should be doing this at all. This really appears to be an XY problem. You're asking "how to do X" without telling us what the problem you're trying to solve is (the Y). _Why_ do you want to set things up this way? A one-time synchronization or to keep both collections in sync?
Cross Data Center Replication (CDCR) is designed to keep two separate collections in sync on an ongoing basis.
If this is a one-time deal, you can manually issue a replication API "fetchindex" command. What I'd do in that case is set up your collection B with each shard having exactly one replica (i.e. a leader and no followers). Do the fetch and verify that your new collection is as you want it then ADDREPLICA to build out your redundancy.
Best,
Erick
On Thu, Jan 4, 2018 at 8:01 AM, Sundaram, Dinesh <Di...@mastercard.com> wrote:
> Thanks Shawn for your prompt response. Assume I have solrcloud A server with 1 node runs on 8983 port and solrcloud B server with 1 node runs on 8983, here I want to synch up the collection between solrcloud A and B using the below replication handler. Is this advisable to use at the solrcloud B ?
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="slave">
> <str name="masterUrl">https://urldefense.proofpoint.com/v2/url?u=http-3A__solrcloudA-3A8983_solr_-24-257Bsolr.core.name-257D_replication&d=DwIFaQ&c=uc5ZRXl8dGLM1RMQwf7xTCjRqXF0jmCF6SP0bDlmMmY&r=gCFZFMR7y0gzhIBFz1lKTqHFMl-3R6gq7ojE0Eam2Eg&m=qBCZxvmkOHW9jt8JM8dVSQJuulIJp3Xk2hXvC5bL7DM&s=xGP-8z2aGBFGrtjIbBMFB6f2cfE4bukyOctAVK_HkyI&e=</str>
> <str name="pollInterval">00:00:20</str>
> </lst>
> </requestHandler>
>
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Shawn Heisey [mailto:apache@elyograg.org]
> Sent: Tuesday, January 2, 2018 5:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solrcloud with Master/Slave
>
> On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
>> I have spun up single solrcloud node on 2 servers.
>
> This makes no sense. If you have two servers, then you probably have more than a single node.
>
>> tried to synch up the data b/w those servers via zookeeper
>
> This is not done with zookeeper. SolrCloud should handle it automatically. SolrCloud uses the zookeeper database to *coordinate* keeping machines in sync, but it's Solr that does the work, not zookeeper.
>
> This makes even less sense when taken in context with the previous sentence. If you only have a single node, then you can't possibly sync between them.
>
>> but didn’t work well due to out of memory issues, ensemble issues
>> with multiple ports connectivity. So had to move to Master slave
>> replication b/w those 2 solrcloud nodes. I couldn’t find any issues
>> so far. Is this advisable? Because I’m wondering that looks like
>> mixing up solrcloud and master/slave replication.
>
> If you're getting OOME problems, then whatever program threw the OOME most likely needs more heap. Or you need to take steps to reduce the amount of heap that's required. Note that this second option might not actually be possible ... increasing the heap is probably the only option you have. Since version 5.0, Solr has shipped with the default heap set to 512MB, which is extremely small. Most users need to increase it.
>
> You can't mix master-slave replication and SolrCloud. SolrCloud takes over the replication feature for its own purposes. Trying to mix these is going to cause you problems. You may not run into the problems immediately, but it is likely that you would run into a problem eventually. Data loss would be possible.
>
> The latest versions of Solr have new SolrCloud replication types that closely mimic the old master-slave replication.
>
> Perhaps you should start over and describe what you've actually seen -- exactly what you've done and configured, and how the results differed from your expectations. Precise commands entered will be helpful.
>
> Thanks,
> Shawn
>
>
> CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
Re: Solrcloud with Master/Slave
Posted by Erick Erickson <er...@gmail.com>.
Whoa. I don't think you should be doing this at all. This really
appears to be an XY problem. You're asking "how to do X" without
telling us what the problem you're trying to solve is (the Y). _Why_
do you want to set things up this way? A one-time synchronization or
to keep both collections in sync?
Cross Data Center Replication (CDCR) is designed to keep two separate
collections in sync on an ongoing basis.
If this is a one-time deal, you can manually issue a replication API
"fetchindex" command. What I'd do in that case is set up your
collection B with each shard having exactly one replica (i.e. a leader
and no followers). Do the fetch and verify that your new collection is
as you want it then ADDREPLICA to build out your redundancy.
Best,
Erick
On Thu, Jan 4, 2018 at 8:01 AM, Sundaram, Dinesh
<Di...@mastercard.com> wrote:
> Thanks Shawn for your prompt response. Assume I have solrcloud A server with 1 node runs on 8983 port and solrcloud B server with 1 node runs on 8983, here I want to synch up the collection between solrcloud A and B using the below replication handler. Is this advisable to use at the solrcloud B ?
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="slave">
> <str name="masterUrl">http://solrcloudA:8983/solr/${solr.core.name}/replication</str>
> <str name="pollInterval">00:00:20</str>
> </lst>
> </requestHandler>
>
>
>
> Dinesh Sundaram
> MBS Platform Engineering
>
> Mastercard
>
>
>
> -----Original Message-----
> From: Shawn Heisey [mailto:apache@elyograg.org]
> Sent: Tuesday, January 2, 2018 5:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Solrcloud with Master/Slave
>
> On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
>> I have spun up single solrcloud node on 2 servers.
>
> This makes no sense. If you have two servers, then you probably have more than a single node.
>
>> tried to synch up the data b/w those servers via zookeeper
>
> This is not done with zookeeper. SolrCloud should handle it automatically. SolrCloud uses the zookeeper database to *coordinate* keeping machines in sync, but it's Solr that does the work, not zookeeper.
>
> This makes even less sense when taken in context with the previous sentence. If you only have a single node, then you can't possibly sync between them.
>
>> but didn’t work well due to out of memory issues, ensemble issues with
>> multiple ports connectivity. So had to move to Master slave
>> replication b/w those 2 solrcloud nodes. I couldn’t find any issues so
>> far. Is this advisable? Because I’m wondering that looks like mixing
>> up solrcloud and master/slave replication.
>
> If you're getting OOME problems, then whatever program threw the OOME most likely needs more heap. Or you need to take steps to reduce the amount of heap that's required. Note that this second option might not actually be possible ... increasing the heap is probably the only option you have. Since version 5.0, Solr has shipped with the default heap set to 512MB, which is extremely small. Most users need to increase it.
>
> You can't mix master-slave replication and SolrCloud. SolrCloud takes over the replication feature for its own purposes. Trying to mix these is going to cause you problems. You may not run into the problems immediately, but it is likely that you would run into a problem eventually. Data loss would be possible.
>
> The latest versions of Solr have new SolrCloud replication types that closely mimic the old master-slave replication.
>
> Perhaps you should start over and describe what you've actually seen -- exactly what you've done and configured, and how the results differed from your expectations. Precise commands entered will be helpful.
>
> Thanks,
> Shawn
>
>
> CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
RE: Solrcloud with Master/Slave
Posted by "Sundaram, Dinesh" <Di...@mastercard.com>.
Thanks Shawn for your prompt response. Assume I have solrcloud A server with 1 node runs on 8983 port and solrcloud B server with 1 node runs on 8983, here I want to synch up the collection between solrcloud A and B using the below replication handler. Is this advisable to use at the solrcloud B ?
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<lst name="slave">
<str name="masterUrl">http://solrcloudA:8983/solr/${solr.core.name}/replication</str>
<str name="pollInterval">00:00:20</str>
</lst>
</requestHandler>
Dinesh Sundaram
MBS Platform Engineering
Mastercard
-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org]
Sent: Tuesday, January 2, 2018 5:33 PM
To: solr-user@lucene.apache.org
Subject: Re: Solrcloud with Master/Slave
On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
> I have spun up single solrcloud node on 2 servers.
This makes no sense. If you have two servers, then you probably have more than a single node.
> tried to synch up the data b/w those servers via zookeeper
This is not done with zookeeper. SolrCloud should handle it automatically. SolrCloud uses the zookeeper database to *coordinate* keeping machines in sync, but it's Solr that does the work, not zookeeper.
This makes even less sense when taken in context with the previous sentence. If you only have a single node, then you can't possibly sync between them.
> but didn’t work well due to out of memory issues, ensemble issues with
> multiple ports connectivity. So had to move to Master slave
> replication b/w those 2 solrcloud nodes. I couldn’t find any issues so
> far. Is this advisable? Because I’m wondering that looks like mixing
> up solrcloud and master/slave replication.
If you're getting OOME problems, then whatever program threw the OOME most likely needs more heap. Or you need to take steps to reduce the amount of heap that's required. Note that this second option might not actually be possible ... increasing the heap is probably the only option you have. Since version 5.0, Solr has shipped with the default heap set to 512MB, which is extremely small. Most users need to increase it.
You can't mix master-slave replication and SolrCloud. SolrCloud takes over the replication feature for its own purposes. Trying to mix these is going to cause you problems. You may not run into the problems immediately, but it is likely that you would run into a problem eventually. Data loss would be possible.
The latest versions of Solr have new SolrCloud replication types that closely mimic the old master-slave replication.
Perhaps you should start over and describe what you've actually seen -- exactly what you've done and configured, and how the results differed from your expectations. Precise commands entered will be helpful.
Thanks,
Shawn
CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
Re: Solrcloud with Master/Slave
Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/2/2018 3:32 PM, Sundaram, Dinesh wrote:
> I have spun up single solrcloud node on 2 servers.
This makes no sense. If you have two servers, then you probably have
more than a single node.
> tried to synch up the data b/w those servers via zookeeper
This is not done with zookeeper. SolrCloud should handle it
automatically. SolrCloud uses the zookeeper database to *coordinate*
keeping machines in sync, but it's Solr that does the work, not zookeeper.
This makes even less sense when taken in context with the previous
sentence. If you only have a single node, then you can't possibly sync
between them.
> but didn’t work well due to out of memory issues, ensemble issues with
> multiple ports connectivity. So had to move to Master slave
> replication b/w those 2 solrcloud nodes. I couldn’t find any issues so
> far. Is this advisable? Because I’m wondering that looks like mixing
> up solrcloud and master/slave replication.
If you're getting OOME problems, then whatever program threw the OOME
most likely needs more heap. Or you need to take steps to reduce the
amount of heap that's required. Note that this second option might not
actually be possible ... increasing the heap is probably the only option
you have. Since version 5.0, Solr has shipped with the default heap set
to 512MB, which is extremely small. Most users need to increase it.
You can't mix master-slave replication and SolrCloud. SolrCloud takes
over the replication feature for its own purposes. Trying to mix these
is going to cause you problems. You may not run into the problems
immediately, but it is likely that you would run into a problem
eventually. Data loss would be possible.
The latest versions of Solr have new SolrCloud replication types that
closely mimic the old master-slave replication.
Perhaps you should start over and describe what you've actually seen --
exactly what you've done and configured, and how the results differed
from your expectations. Precise commands entered will be helpful.
Thanks,
Shawn