You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Francisco Nogueira Calmon Sobral <fs...@igcorp.com.br> on 2013/11/07 14:08:45 UTC
sstableloader in Counter Columns
Hi, all!
I've performed a test in my cluster, regarding the sstableloader behaviour on counter column families. The test cluster has 3 nodes running Cassandra 1.2.3 with RF=3. The machine that sstableloader run had Cassandra 1.2.11.
The counter column family had only one row, so I chose one node and performed nodetool snapshot. Then, I've truncated this CF.
Before truncating, this CF was:
RowKey: 0
=> (counter=EVENTS, value=4816777)
=> (counter=ITEMS, value=395930)
=> (counter=USERS, value=2574764)
After truncating and running sstableloader it was:
[default@Sessions] get Counters[0];
=> (counter=EVENTS, value=4809758)
=> (counter=ITEMS, value=382473)
=> (counter=USERS, value=2571674)
Then, I've performed another test. I've truncated the CF, then incremented the row 'ITEMS' by 1.000.000 and then run sstableloader. Unfortunately, the result was:
[default@Sessions] get Counters[0];
=> (counter=EVENTS, value=4809758)
=> (counter=ITEMS, value=382473)
=> (counter=USERS, value=2571674)
Is this the normal behaviour of sstableloader for counter column families? If that is the case, we cannot run sstableloader in a live cluster, can we?
Best regards!
Francisco
Re: sstableloader in Counter Columns
Posted by Aaron Morton <aa...@thelastpickle.com>.
> After truncating and running sstableloader it was:
>
> [default@Sessions] get Counters[0];
> => (counter=EVENTS, value=4809758)
> => (counter=ITEMS, value=382473)
> => (counter=USERS, value=2571674)
The expected value was 395930 and you got 382473 instead ?
Was there any errors / warnings in the log about counter shards ?
> Then, I've performed another test. I've truncated the CF, then incremented the row 'ITEMS' by 1.000.000 and then run sstableloader. Unfortunately, the result was:
>
> [default@Sessions] get Counters[0];
> => (counter=EVENTS, value=4809758)
> => (counter=ITEMS, value=382473)
> => (counter=USERS, value=2571674)
What was the expected value ?
> The counter column family had only one row, so I chose one node and performed nodetool snapshot. Then, I've truncated this CF.
The counter value is broken up into shards, each node that is a replica for the row maintains it’s own shard and replicates that to the other nodes.
If your cluster was dropped messages and you took the snapshot from only one machine it’s possible you did not get a consistenct view of the data.
> Is this the normal behaviour of sstableloader for counter column families? If that is the case, we cannot run sstableloader in a live cluster, can we?
I would be interested to know if you get the same result when using snapshots from all nodes.
Cheers
-----------------
Aaron Morton
New Zealand
@aaronmorton
Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
On 8/11/2013, at 2:08 am, Francisco Nogueira Calmon Sobral <fs...@igcorp.com.br> wrote:
> Hi, all!
>
> I've performed a test in my cluster, regarding the sstableloader behaviour on counter column families. The test cluster has 3 nodes running Cassandra 1.2.3 with RF=3. The machine that sstableloader run had Cassandra 1.2.11.
>
> The counter column family had only one row, so I chose one node and performed nodetool snapshot. Then, I've truncated this CF.
>
> Before truncating, this CF was:
>
> RowKey: 0
> => (counter=EVENTS, value=4816777)
> => (counter=ITEMS, value=395930)
> => (counter=USERS, value=2574764)
>
> After truncating and running sstableloader it was:
>
> [default@Sessions] get Counters[0];
> => (counter=EVENTS, value=4809758)
> => (counter=ITEMS, value=382473)
> => (counter=USERS, value=2571674)
>
> Then, I've performed another test. I've truncated the CF, then incremented the row 'ITEMS' by 1.000.000 and then run sstableloader. Unfortunately, the result was:
>
> [default@Sessions] get Counters[0];
> => (counter=EVENTS, value=4809758)
> => (counter=ITEMS, value=382473)
> => (counter=USERS, value=2571674)
>
>
>
> Is this the normal behaviour of sstableloader for counter column families? If that is the case, we cannot run sstableloader in a live cluster, can we?
>
> Best regards!
> Francisco
>
>