You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by SEGALIS Morgan <ms...@gmail.com> on 2015/01/22 18:36:08 UTC
Does nodetool repair stop the node to answer requests ?
I have been searching all over documentation but could not find an straight
answer.
For a project I'm using a single node cassandra database (so far)... It has
always worked well, but I'm reading everywhere that I should do a nodetool
repair at least every week, especially if I delete rows, which I do.
The issue is, of course I can bring down the unique node, otherwise the
service will be down, and that's unfortunately not an option.
So I wondered, does a nodetool repair make the server stop serving
requests, or does it just use a lot of ressources but still serves request ?
--
Morgan SEGALIS
Re: Does nodetool repair stop the node to answer requests ?
Posted by Ryan Svihla <rs...@foundev.pro>.
For the cases where repair brings down a cluster, I can safely say that
cluster has more problems than just repair. Consider the load caused by
repair to be somewhat equivalent to losing a node. This is not a 1 for 1
comparison, as the node you're running repair on is up albeit busy,and the
potential streaming impacts can be of course worse, however, the entire
design of Cassandra is around being able to tolerate 1 or more node outages
(depending on your level of sizing) and not have your application be
affected.
On Thu, Jan 22, 2015 at 1:15 PM, SEGALIS Morgan <ms...@gmail.com> wrote:
> Don't think it is near failure, it uses only 3% of the CPU and 40% of the
> RAM if that is what you meant.
>
> 2015-01-22 19:58 GMT+01:00 Robert Coli <rc...@eventbrite.com>:
>
>> On Thu, Jan 22, 2015 at 10:53 AM, SEGALIS Morgan <ms...@gmail.com>
>> wrote:
>>
>>> what do you mean by "operating correctly" ?
>>>
>>
>> I mean that if you are operating near failure, repair might trip a node
>> into failure. But if you are operating correctly, repair should not.
>>
>> =Rob
>>
>>
>
>
>
> --
> Morgan SEGALIS
>
--
Thanks,
Ryan Svihla
Re: Does nodetool repair stop the node to answer requests ?
Posted by SEGALIS Morgan <ms...@gmail.com>.
Don't think it is near failure, it uses only 3% of the CPU and 40% of the
RAM if that is what you meant.
2015-01-22 19:58 GMT+01:00 Robert Coli <rc...@eventbrite.com>:
> On Thu, Jan 22, 2015 at 10:53 AM, SEGALIS Morgan <ms...@gmail.com>
> wrote:
>
>> what do you mean by "operating correctly" ?
>>
>
> I mean that if you are operating near failure, repair might trip a node
> into failure. But if you are operating correctly, repair should not.
>
> =Rob
>
>
--
Morgan SEGALIS
Re: Does nodetool repair stop the node to answer requests ?
Posted by Robert Coli <rc...@eventbrite.com>.
On Thu, Jan 22, 2015 at 10:53 AM, SEGALIS Morgan <ms...@gmail.com> wrote:
> what do you mean by "operating correctly" ?
>
I mean that if you are operating near failure, repair might trip a node
into failure. But if you are operating correctly, repair should not.
=Rob
Re: Does nodetool repair stop the node to answer requests ?
Posted by SEGALIS Morgan <ms...@gmail.com>.
what do you mean by "operating correctly" ?
I only use dynamic columns if that helps...
2015-01-22 19:10 GMT+01:00 Robert Coli <rc...@eventbrite.com>:
> On Thu, Jan 22, 2015 at 9:36 AM, SEGALIS Morgan <ms...@gmail.com>
> wrote:
>
>> So I wondered, does a nodetool repair make the server stop serving
>> requests, or does it just use a lot of ressources but still serves request ?
>>
>
> In pathological cases, repair can cause a node to seriously degrade. If
> you are operating correctly, it just uses lots of resources but still
> serves requests.
>
> =Rob
> http://twitter.com/rcolidba
>
--
Morgan SEGALIS
Re: Does nodetool repair stop the node to answer requests ?
Posted by Tim Heckman <ti...@pagerduty.com>.
On Thu, Jan 22, 2015 at 10:22 AM, Jan <cn...@yahoo.com> wrote:
> Running a 'nodetool repair' will 'not' bring the node down.
It's not something that happens during normal operation. If something
goes sideways, and the resource usage climbs, a repair can definitely
cripple a node.
> Your question:
> does a nodetool repair make the server stop serving requests, or does it
> just use a lot of ressources but still serves request
>
> Answer: NO, the server will not stop serving requests.
> It will use some resources but not enough to affect the server serving
> requests.
I don't think this is right. I've personally seen repair operations
cause real bad things to happen to an entire Cassandra cluster. The
only mitigation was to shut that misbehaving node down and then normal
operations continued within the cluster.
> hope this helps
> Jan
Cheers!
-Tim
Re: Does nodetool repair stop the node to answer requests ?
Posted by SEGALIS Morgan <ms...@gmail.com>.
Thanks, this is a straight forward answer, exactly what I needed !
2015-01-22 19:22 GMT+01:00 Jan <cn...@yahoo.com>:
> Running a 'nodetool repair' will 'not' bring the node down.
>
> Your question:
> does a nodetool repair make the server stop serving requests, or does it
> just use a lot of ressources but still serves request
>
> Answer: NO, the server will not stop serving requests.
> It will use some resources but not enough to affect the server serving
> requests.
>
> hope this helps
> Jan
>
>
>
--
Morgan SEGALIS
Re: Does nodetool repair stop the node to answer requests ?
Posted by Jan <cn...@yahoo.com>.
Running a 'nodetool repair' will 'not' bring the node down.
Your question: does a nodetool repair make the server stop serving requests, or does it just use a lot of ressources but still serves request
Answer: NO, the server will not stop serving requests. It will use some resources but not enough to affect the server serving requests.
hope this helpsJan
Re: Does nodetool repair stop the node to answer requests ?
Posted by SEGALIS Morgan <ms...@gmail.com>.
If I change the network topology, I have to run repair right before adding
a new cluster ?
I know that I should put 2 more nodes, so far, I'm preparing myself to
create a new node on a new DC, but on same network (ping really low), so at
least I would have a backup server if anything happens, just need to wrap
my head around the network topology and understand what I do before
starting a new node.
2015-01-22 19:15 GMT+01:00 Flavien Charlon <fl...@gmail.com>:
> I don't think you can do nodetool repair on a single node cluster.
>
> Still, one day or another you'll have to reboot your server, at which
> point your cluster will be down. If you want high availability, you should
> use a 3 nodes cluster with RF = 3.
>
> On 22 January 2015 at 18:10, Robert Coli <rc...@eventbrite.com> wrote:
>
>> On Thu, Jan 22, 2015 at 9:36 AM, SEGALIS Morgan <ms...@gmail.com>
>> wrote:
>>
>>> So I wondered, does a nodetool repair make the server stop serving
>>> requests, or does it just use a lot of ressources but still serves request ?
>>>
>>
>> In pathological cases, repair can cause a node to seriously degrade. If
>> you are operating correctly, it just uses lots of resources but still
>> serves requests.
>>
>> =Rob
>> http://twitter.com/rcolidba
>>
>
>
--
Morgan SEGALIS
Re: Does nodetool repair stop the node to answer requests ?
Posted by Flavien Charlon <fl...@gmail.com>.
I don't think you can do nodetool repair on a single node cluster.
Still, one day or another you'll have to reboot your server, at which point
your cluster will be down. If you want high availability, you should use a
3 nodes cluster with RF = 3.
On 22 January 2015 at 18:10, Robert Coli <rc...@eventbrite.com> wrote:
> On Thu, Jan 22, 2015 at 9:36 AM, SEGALIS Morgan <ms...@gmail.com>
> wrote:
>
>> So I wondered, does a nodetool repair make the server stop serving
>> requests, or does it just use a lot of ressources but still serves request ?
>>
>
> In pathological cases, repair can cause a node to seriously degrade. If
> you are operating correctly, it just uses lots of resources but still
> serves requests.
>
> =Rob
> http://twitter.com/rcolidba
>
Re: Does nodetool repair stop the node to answer requests ?
Posted by Robert Coli <rc...@eventbrite.com>.
On Thu, Jan 22, 2015 at 9:36 AM, SEGALIS Morgan <ms...@gmail.com> wrote:
> So I wondered, does a nodetool repair make the server stop serving
> requests, or does it just use a lot of ressources but still serves request ?
>
In pathological cases, repair can cause a node to seriously degrade. If you
are operating correctly, it just uses lots of resources but still serves
requests.
=Rob
http://twitter.com/rcolidba