easy repair questions on -pr

["Hiller, Dean" <De...@nrel.gov>]

If I understand –pr correctly…

 1.  -pr forces only the current nodes' stables to be fixed (so I run on each node once)
 2.  Can I run node tool –pr repair on just 1/RF of my nodes if I do the correct nodes?
 3.  Without the –pr, it will fix all the stuff on the current node AND the nodes with replicas?
 4.  But is it true replicas may have other data that still needs repair as well so I would still need to go to those nodes and run ./nodetool –pr repair?
 5.  I ran nodetool –pr on EVERY node, but I still see rows that were deleted…well, I see their keys only so the columns appear to be deleted correctly.  Why are the row keys still there though? (the cassandra-cli and cqlsh both have the same results).
 6.  The below is very confusing where it says "only first range returned by partitioner for a node is repaired".  This makes it sound like only a range of keys is repaired on the current node and the other rows on the current node are not touched.  How to correctly read that sentence???  Would it be more accurate to say "All the rows that this node is responsible for are repaired" vs. not using –pr means whole cluster is repaired or is that incorrect?

Begins an anti-entropy node repair operation. If the -pr option is specified, only the first range returned by the partitioner for a node is repaired. This allows you to repair each node in the cluster in succession without duplicating work.

Without -pr, all replica ranges that the node is responsible for are repaired.

Optionally takes a list of column family names.

Re: easy repair questions on -pr

[Sylvain Lebresne <sy...@datastax.com>]

> So, I suspect I can expect those rows to finally go away when queried from
> cassandra-cli once GCGraceSeconds has passed then?

Yes.

--
Sylvain

Re: easy repair questions on -pr

["Hiller, Dean" <De...@nrel.gov>]

GREAT answer, thanks and one last questionŠ

So, I suspect I can expect those rows to finally go away when queried from
cassandra-cli once GCGraceSeconds has passed then?

Or will they always be there forever and ever and ever(this can't be true,
right).

Thanks,
Dean

On 10/2/12 9:34 AM, "Sylvain Lebresne" <sy...@datastax.com> wrote:

>The short version is: there is 2 use case for nodetool repair:
>  1) For periodic repair of the whole cluster
>(http://wiki.apache.org/cassandra/Operations#Frequency_of_nodetool_repair)
>.
>In that case, you should run repair *with* -pr and you should run it
>on *every* node.
>  2) When a node has been down for a long time (for instance long
>enough that hints may have been dropped), and you want to repair that
>node specifically. In that case, you should run repair on that node
>only and you should use it *without* -pr.
>
>As for the gory details, nodetool repair without -pr will repair all
>the range of the node on which the repair is done. But when a range is
>repaired, it is repaired on *all* replica. In other words, a repair on
>node A will also repair parts of other nodes that share a range with
>A. That why, in the case 1) above, where you want to repair the whole
>cluster, a repair without -pr is inefficient, because if you repair A
>and B and both are replica for the same range, you will duplicate the
>work. Hence repair -pr: on one node it repair only its primary range
>(but all replica for said range), and so if you do that on every node,
>you will have effectively repair the whole cluster without having
>repaired the same range twice.
>
>> Can I run node tool ­pr repair on just 1/RF of my nodes if I do the
>>correct nodes?
>
>As it's hopefully clear from the description above, no.
>
>>  Why are the row keys still there though?
>
>http://wiki.apache.org/cassandra/FAQ#range_ghosts
>
>
>--
>Sylvain

Re: easy repair questions on -pr

[Sylvain Lebresne <sy...@datastax.com>]

The short version is: there is 2 use case for nodetool repair:
  1) For periodic repair of the whole cluster
(http://wiki.apache.org/cassandra/Operations#Frequency_of_nodetool_repair).
In that case, you should run repair *with* -pr and you should run it
on *every* node.
  2) When a node has been down for a long time (for instance long
enough that hints may have been dropped), and you want to repair that
node specifically. In that case, you should run repair on that node
only and you should use it *without* -pr.

As for the gory details, nodetool repair without -pr will repair all
the range of the node on which the repair is done. But when a range is
repaired, it is repaired on *all* replica. In other words, a repair on
node A will also repair parts of other nodes that share a range with
A. That why, in the case 1) above, where you want to repair the whole
cluster, a repair without -pr is inefficient, because if you repair A
and B and both are replica for the same range, you will duplicate the
work. Hence repair -pr: on one node it repair only its primary range
(but all replica for said range), and so if you do that on every node,
you will have effectively repair the whole cluster without having
repaired the same range twice.

> Can I run node tool –pr repair on just 1/RF of my nodes if I do the correct nodes?

As it's hopefully clear from the description above, no.

>  Why are the row keys still there though?

http://wiki.apache.org/cassandra/FAQ#range_ghosts


--
Sylvain