You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Peter Haggerty <pe...@librato.com> on 2014/10/16 07:07:28 UTC

Should a node that is bootstrapping be receiving writes in addition to the streams it is receiving?

We bootstrapped a node with replace_address and noticed that the
WriteCount for each CF stayed at 0 until the bootstrap was complete.
After the bootstrap completed cfstats reported the expected values for
WriteCount on each CF.

The node wrote gigs of data to various CFs during the bootstrap so it
was clearly "writing" in some sense and it has the expected behavior
after the bootstrap. Is cfstats correct when it reports that there
were no writes during a bootstrap?


Peter

Re: Should a node that is bootstrapping be receiving writes in addition to the streams it is receiving?

Posted by Robert Coli <rc...@eventbrite.com>.
On Mon, Mar 2, 2015 at 1:58 PM, Paulo Ricardo Motta Gomes <
paulo.motta@chaordicsystems.com> wrote:

> I also checked via JMX and all the write counts are zero. Is the node
> supposed to receive writes during bootstrap?
>

As I understand it, yes.

The other funny thing during bootstrap, is that nodetool status shows that
> the bootsrapping node is Up/Normal (UN), instead of Up/Joining(UJ), is this
> expected or is it a bug? The bootstrapping node does not even appear in the
> "nodetool status" of other nodes.
>

Perhaps this node is not actually bootstrapping because you have configured
it as a seed with no other valid seeds listed and so it has started as a
cluster of one?

=Rob

Re: Should a node that is bootstrapping be receiving writes in addition to the streams it is receiving?

Posted by Paulo Ricardo Motta Gomes <pa...@chaordicsystems.com>.
I'm also facing a similar issue while bootstrapping a replacement node via
-Dreplace_address flag. The node is streaming data from neighbors, but
cfstats shows 0 counts for all metrics of all CFs in the bootstrapping node:

        SSTable count: 0
        SSTables in each level: [0, 0, 0, 0, 0, 0, 0, 0, 0]
        Space used (live), bytes: 0
        Space used (total), bytes: 0
        SSTable Compression Ratio: 0.0
        Number of keys (estimate): 0
        Memtable cell count: 0
        Memtable data size, bytes: 0
        Memtable switch count: 0
        Local read count: 0
        Local read latency: 0.000 ms
        Local write count: 0
        Local write latency: 0.000 ms
        Pending tasks: 0
        Bloom filter false positives: 0
        Bloom filter false ratio: 0.00000
        Bloom filter space used, bytes: 0
        Compacted partition minimum bytes: 0
        Compacted partition maximum bytes: 0
        Compacted partition mean bytes: 0
        Average live cells per slice (last five minutes): 0.0
        Average tombstones per slice (last five minutes): 0.0

I also checked via JMX and all the write counts are zero. Is the node
supposed to receive writes during bootstrap?

The other funny thing during bootstrap, is that nodetool status shows that
the bootsrapping node is Up/Normal (UN), instead of Up/Joining(UJ), is this
expected or is it a bug? The bootstrapping node does not even appear in the
"nodetool status" of other nodes.

UN  X.Y.Z.244  15.9 GB    1       3.7%
52fb21e-4621-4533-b201-8c1a7adbe818  rack

If I do a nodetool netstats, I see:

Mode: JOINING
Bootstrap 647d4b30-c11e-11e4-9249-173e73521fb44

Cheers,

Paulo

On Thu, Oct 16, 2014 at 3:53 PM, Robert Coli <rc...@eventbrite.com> wrote:

> On Wed, Oct 15, 2014 at 10:07 PM, Peter Haggerty <
> peter.haggerty@librato.com> wrote:
>
>> The node wrote gigs of data to various CFs during the bootstrap so it
>> was clearly "writing" in some sense and it has the expected behavior
>> after the bootstrap. Is cfstats correct when it reports that there
>> were no writes during a bootstrap?
>>
>
> As I understand it :
>
> Writes ("extra" writes, from the perspective of replication factor, f/e a
> RF=3 cluster has effective RF=4 during bootstrap, but not relevant for
> consistency purposes until end of bootstrap) occur via the storage protocol
> during bootstrap, so I would expect to see those reflected in cfstats.
>
> I'm relatively confident it is in fact receiving those writes, so your
> confusion might just be a result of how it's reported?
>
> =Rob
> http://twitter.com/rcolidba
>
>


-- 
*Paulo Motta*

Chaordic | *Platform*
*www.chaordic.com.br <http://www.chaordic.com.br/>*
+55 48 3232.3200

Re: Should a node that is bootstrapping be receiving writes in addition to the streams it is receiving?

Posted by Robert Coli <rc...@eventbrite.com>.
On Wed, Oct 15, 2014 at 10:07 PM, Peter Haggerty <peter.haggerty@librato.com
> wrote:

> The node wrote gigs of data to various CFs during the bootstrap so it
> was clearly "writing" in some sense and it has the expected behavior
> after the bootstrap. Is cfstats correct when it reports that there
> were no writes during a bootstrap?
>

As I understand it :

Writes ("extra" writes, from the perspective of replication factor, f/e a
RF=3 cluster has effective RF=4 during bootstrap, but not relevant for
consistency purposes until end of bootstrap) occur via the storage protocol
during bootstrap, so I would expect to see those reflected in cfstats.

I'm relatively confident it is in fact receiving those writes, so your
confusion might just be a result of how it's reported?

=Rob
http://twitter.com/rcolidba