You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Philippe <wa...@gmail.com> on 2011/08/17 08:57:37 UTC

Bulk loading into live data

http://www.datastax.com/dev/blog/bulk-loading indicates that "it is
perfectly reasonable to load data into a live, active cluster."

So lets say my cluster has a single KS & CF and it contains a key "test"
with a SC named "Cass" and a normal subcolumn named "Data" that has value 1.

If I SSTLoad data into that KS & CF that has the same key, it will rely on
timestamps stored in the SSTable to overwrite value "1" or not, right ?
What if the column is a counter ? Does it overwrite or increment ? Ie if the
SST I am loading has the exact same setup but value 2, will my value change
to 3 ?

Thanks

Re: Bulk loading into live data

Posted by aaron morton <aa...@thelastpickle.com>.
That is my understanding. 

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 18/08/2011, at 12:36 AM, Philippe wrote:

>> What if the column is a counter ? Does it overwrite or increment ? Ie if the SST I am loading has the exact same setup but value 2, will my value change to 3 ?
> Counter columns only know how to increment (assuming no deletes), so you will get 3. See https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java#L153 and https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/CounterColumn.java#L138
> Great, so I assume that I can "add" the same counter column multiple times for the same key/SC in different nextRow() calls and the counter will still get incremented ?
> My use-case is that my bulk loading has data that's aggregated at a finer granularity than the data in cassandra.
> 
> Thanks
>

Re: Bulk loading into live data

Posted by Philippe <wa...@gmail.com>.
>
> What if the column is a counter ? Does it overwrite or increment ? Ie if
> the SST I am loading has the exact same setup but value 2, will my value
> change to 3 ?
>
> Counter columns only know how to increment (assuming no deletes), so you
> will get 3. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java#L153
>  and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/CounterColumn.java#L138
>
Great, so I assume that I can "add" the same counter column multiple times
for the same key/SC in different nextRow() calls and the counter will still
get incremented ?
My use-case is that my bulk loading has data that's aggregated at a finer
granularity than the data in cassandra.

Thanks

Re: Bulk loading into live data

Posted by aaron morton <aa...@thelastpickle.com>.
> If I SSTLoad data into that KS & CF that has the same key, it will rely on timestamps stored in the SSTable to overwrite value "1" or not, right ?
yes. 
> What if the column is a counter ? Does it overwrite or increment ? Ie if the SST I am loading has the exact same setup but value 2, will my value change to 3 ?
Counter columns only know how to increment (assuming no deletes), so you will get 3. See https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java#L153 and https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/CounterColumn.java#L138

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 17/08/2011, at 6:57 PM, Philippe wrote:

> http://www.datastax.com/dev/blog/bulk-loading indicates that "it is perfectly reasonable to load data into a live, active cluster."
> 
> So lets say my cluster has a single KS & CF and it contains a key "test" with a SC named "Cass" and a normal subcolumn named "Data" that has value 1.
> 
> If I SSTLoad data into that KS & CF that has the same key, it will rely on timestamps stored in the SSTable to overwrite value "1" or not, right ?
> What if the column is a counter ? Does it overwrite or increment ? Ie if the SST I am loading has the exact same setup but value 2, will my value change to 3 ?
> 
> Thanks