You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Nazik Huq <na...@yahoo.com> on 2014/03/07 00:15:43 UTC
SolrCloud recovery after nodes are rebooted in rapid succession
Hello,
I have a question from a colleague who's managing a 3-node(VMs) SolrCloud
cluster with a separate 3-node Zookeeper ensemble. Periodically the data
center underneath the SolrCloud decides to upgrade the SolrCloud instance
infrastructure in a "rolling upgrade" fashion. So after the 1st instance of
the SolrCloud is shut down and while it is in the process of rebooting, the
2nd instance starts to shut down and so on. Eventually all three Solr
instances are rebooted and up and running but the cluster in now
inoperable. Meaning clients can't query or ingest data. My colleague is
trying to ascertain if this problem is due to Solr's inability to recover
from a rapid succession of reboots of the nodes or from the data center
upgrade that is triggering a "situation" making SolrCloud inoperable.
My question is, can a SolrCloud cluster become inoperable after its nodes
are rebooted in rapid succession as described above? Is there an edge case
similar to this?
Thanks,
Nazik Huq
RE: SolrCloud recovery after nodes are rebooted in rapid succession
Posted by Nazik Huq <na...@yahoo.com>.
The version is 4.6. I am going to ask for the log files and post it.
-----Original Message-----
From: Mark Miller [mailto:markrmiller@gmail.com]
Sent: Thursday, March 06, 2014 6:33 PM
To: solr-user
Subject: Re: SolrCloud recovery after nodes are rebooted in rapid succession
Would probably need to see some logs to say much. Need to understand why
they are inoperable.
What version is this?
- Mark
http://about.me/markrmiller
On Mar 6, 2014, at 6:15 PM, Nazik Huq <na...@yahoo.com> wrote:
> Hello,
>
>
>
> I have a question from a colleague who's managing a 3-node(VMs)
> SolrCloud cluster with a separate 3-node Zookeeper ensemble.
> Periodically the data center underneath the SolrCloud decides to
> upgrade the SolrCloud instance infrastructure in a "rolling upgrade"
> fashion. So after the 1st instance of the SolrCloud is shut down and
> while it is in the process of rebooting, the 2nd instance starts to
> shut down and so on. Eventually all three Solr instances are rebooted
> and up and running but the cluster in now inoperable. Meaning
> clients can't query or ingest data. My colleague is trying to
> ascertain if this problem is due to Solr's inability to recover from a
> rapid succession of reboots of the nodes or from the data center upgrade
that is triggering a "situation" making SolrCloud inoperable.
>
>
>
> My question is, can a SolrCloud cluster become inoperable after its
> nodes are rebooted in rapid succession as described above? Is there an
> edge case similar to this?
>
>
>
> Thanks,
>
>
>
> Nazik Huq
>
>
>
>
>
Re: SolrCloud recovery after nodes are rebooted in rapid succession
Posted by Mark Miller <ma...@gmail.com>.
Would probably need to see some logs to say much. Need to understand why they are inoperable.
What version is this?
- Mark
http://about.me/markrmiller
On Mar 6, 2014, at 6:15 PM, Nazik Huq <na...@yahoo.com> wrote:
> Hello,
>
>
>
> I have a question from a colleague who's managing a 3-node(VMs) SolrCloud
> cluster with a separate 3-node Zookeeper ensemble. Periodically the data
> center underneath the SolrCloud decides to upgrade the SolrCloud instance
> infrastructure in a "rolling upgrade" fashion. So after the 1st instance of
> the SolrCloud is shut down and while it is in the process of rebooting, the
> 2nd instance starts to shut down and so on. Eventually all three Solr
> instances are rebooted and up and running but the cluster in now
> inoperable. Meaning clients can't query or ingest data. My colleague is
> trying to ascertain if this problem is due to Solr's inability to recover
> from a rapid succession of reboots of the nodes or from the data center
> upgrade that is triggering a "situation" making SolrCloud inoperable.
>
>
>
> My question is, can a SolrCloud cluster become inoperable after its nodes
> are rebooted in rapid succession as described above? Is there an edge case
> similar to this?
>
>
>
> Thanks,
>
>
>
> Nazik Huq
>
>
>
>
>