Bringing down ZK without Solr

[Venkateswarlu Bommineni <bv...@gmail.com>]

Hi Team,

Is there any way we can bring down ZK without impacting Solr ?

I know it might be a silly question as Solr tolly depends in ZK for all I/O
operations and configuration changes.

Thanks,
Venkat.

Re: Bringing down ZK without Solr

[Vivek Pathak <vp...@orgmeta.com>]

Why need to bring down.  How about bring down network access e.g. by adding a temp firewall rule. 

Or just send a stop signal to zookeeper process.  On test done send a continue 

Sent from my iPhone

> On Jun 9, 2017, at 9:33 AM, Venkateswarlu Bommineni <bv...@gmail.com> wrote:
> 
> Thanks for your reply Eric.
> 
> The use case is We have a script that will send a mail when Solr and ZK
> don't talk to each other.
> 
> so we want to just replicate the issue and test that script.
> 
> but actually, we don't want to bring down Solr and ZK nodes but want to
> just try by disconnecting both of them.
> 
> Thanks,
> Venkat.
> 
> Thanks,
> Venkat.
> 
> 
> 
> On Thu, Jun 8, 2017 at 10:16 PM, Erick Erickson <er...@gmail.com>
> wrote:
> 
>> Well, it depends on what you mean by "impacting".
>> 
>> When ZK drops below quorum you will no longer be able to send indexing
>> requests to Solr, they'll all fail. At least they better ;).
>> 
>> _Queries_ should continue to work, but you're in somewhat uncharted
>> territory, nobody I know runs that way very long ;).
>> 
>> The other thing I'd be sure to test is how robust reconnection is from
>> Solr to ZK when you bring the nodes back up.
>> 
>> bq:  Solr tolly depends in ZK for all I/O
>> 
>> This is a common misunderstanding. Solr depends on ZK for all changes
>> in cluster state, i.e. nodes going up/down/changing state (down,
>> recovering, active etc). Those changes generate traffic between ZK and
>> Solr.
>> 
>> For a normal I/O request, each Solr node has been notified by ZK of
>> the current state of the collection already and that information
>> cached locally. So each node knows everything it needs to know to
>> service the index or query request without talking to ZooKeeper at
>> all.
>> 
>> I know of installations indexing 100s of K documents each second.
>> Actually the record I know of is over 1M docs/second. If each of those
>> requests had to touch ZK to complete ZK could never keep up....
>> 
>> Best,
>> Erick
>> 
>> On Thu, Jun 8, 2017 at 8:49 AM, Venkateswarlu Bommineni
>> <bv...@gmail.com> wrote:
>>> Hi Team,
>>> 
>>> Is there any way we can bring down ZK without impacting Solr ?
>>> 
>>> I know it might be a silly question as Solr tolly depends in ZK for all
>> I/O
>>> operations and configuration changes.
>>> 
>>> Thanks,
>>> Venkat.
>> 

Re: Bringing down ZK without Solr

[Venkateswarlu Bommineni <bv...@gmail.com>]

Thanks for your reply Eric.

The use case is We have a script that will send a mail when Solr and ZK
don't talk to each other.

so we want to just replicate the issue and test that script.

but actually, we don't want to bring down Solr and ZK nodes but want to
just try by disconnecting both of them.

Thanks,
Venkat.

Thanks,
Venkat.



On Thu, Jun 8, 2017 at 10:16 PM, Erick Erickson <er...@gmail.com>
wrote:

> Well, it depends on what you mean by "impacting".
>
> When ZK drops below quorum you will no longer be able to send indexing
> requests to Solr, they'll all fail. At least they better ;).
>
> _Queries_ should continue to work, but you're in somewhat uncharted
> territory, nobody I know runs that way very long ;).
>
> The other thing I'd be sure to test is how robust reconnection is from
> Solr to ZK when you bring the nodes back up.
>
> bq:  Solr tolly depends in ZK for all I/O
>
> This is a common misunderstanding. Solr depends on ZK for all changes
> in cluster state, i.e. nodes going up/down/changing state (down,
> recovering, active etc). Those changes generate traffic between ZK and
> Solr.
>
> For a normal I/O request, each Solr node has been notified by ZK of
> the current state of the collection already and that information
> cached locally. So each node knows everything it needs to know to
> service the index or query request without talking to ZooKeeper at
> all.
>
> I know of installations indexing 100s of K documents each second.
> Actually the record I know of is over 1M docs/second. If each of those
> requests had to touch ZK to complete ZK could never keep up....
>
> Best,
> Erick
>
> On Thu, Jun 8, 2017 at 8:49 AM, Venkateswarlu Bommineni
> <bv...@gmail.com> wrote:
> > Hi Team,
> >
> > Is there any way we can bring down ZK without impacting Solr ?
> >
> > I know it might be a silly question as Solr tolly depends in ZK for all
> I/O
> > operations and configuration changes.
> >
> > Thanks,
> > Venkat.
>

Re: Bringing down ZK without Solr

[Erick Erickson <er...@gmail.com>]

Well, it depends on what you mean by "impacting".

When ZK drops below quorum you will no longer be able to send indexing
requests to Solr, they'll all fail. At least they better ;).

_Queries_ should continue to work, but you're in somewhat uncharted
territory, nobody I know runs that way very long ;).

The other thing I'd be sure to test is how robust reconnection is from
Solr to ZK when you bring the nodes back up.

bq:  Solr tolly depends in ZK for all I/O

This is a common misunderstanding. Solr depends on ZK for all changes
in cluster state, i.e. nodes going up/down/changing state (down,
recovering, active etc). Those changes generate traffic between ZK and
Solr.

For a normal I/O request, each Solr node has been notified by ZK of
the current state of the collection already and that information
cached locally. So each node knows everything it needs to know to
service the index or query request without talking to ZooKeeper at
all.

I know of installations indexing 100s of K documents each second.
Actually the record I know of is over 1M docs/second. If each of those
requests had to touch ZK to complete ZK could never keep up....

Best,
Erick

On Thu, Jun 8, 2017 at 8:49 AM, Venkateswarlu Bommineni
<bv...@gmail.com> wrote:
> Hi Team,
>
> Is there any way we can bring down ZK without impacting Solr ?
>
> I know it might be a silly question as Solr tolly depends in ZK for all I/O
> operations and configuration changes.
>
> Thanks,
> Venkat.