You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Jabbar Azam <aj...@gmail.com> on 2014/05/08 15:07:00 UTC

idempotent counters

Hello,

Do people use counters when they want to have idempotent operations in
cassandra?

I have a use case for using a counter to check for a count of objects in a
partition. If the counter is more than some value then the data in the
partition is moved into two different partitions. I can't work out how to
do this splitting and recover if a problem happens during modification of
the counter.

http://www.ebaytechblog.com/2012/08/14/cassandra-data-modeling-best-practices-part-2explains
that counters shouldn't be used if you want idempotency. I would
agree, but the alternative  is not very elegant. I would have to manully
count the objects in a partition and then move the data and repeat the
operation if something went wrong.

It is less resource intensive to read a counter value to see if a partition
needs splitting then to read all the objects in a partition. The counter
value can be stored in its own table sorting in descending order of the
counter value.

Does anybody else use another technique for achieving this idempotency with
counters?

I'm using cassandra 2.0.7.



Thanks

Jabbar Azam

Re: idempotent counters

Posted by Jabbar Azam <aj...@gmail.com>.

Thanks Aaron. I've mitigated this by removing the dependency on idempotent
counters. But its good to know the limitations of counters.

Thanks

Jabbar Azam
On 19 May 2014 08:36, "Aaron Morton" <aa...@thelastpickle.com> wrote:

> Does anybody else use another technique for achieving this idempotency
> with counters?
>
>
> The idempotency problem with counters has to do with what will happen when
> you get a timeout. If you reply the write there is a chance of the
> increment been applied twice. This is inherent in the current design.
>
> Cheers
> Aaron
>
>
> -----------------
> Aaron Morton
> New Zealand
> @aaronmorton
>
> Co-Founder & Principal Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> On 9/05/2014, at 1:07 am, Jabbar Azam <aj...@gmail.com> wrote:
>
> Hello,
>
> Do people use counters when they want to have idempotent operations in
> cassandra?
>
> I have a use case for using a counter to check for a count of objects in a
> partition. If the counter is more than some value then the data in the
> partition is moved into two different partitions. I can't work out how to
> do this splitting and recover if a problem happens during modification of
> the counter.
>
>
> http://www.ebaytechblog.com/2012/08/14/cassandra-data-modeling-best-practices-part-2explains that counters shouldn't be used if you want idempotency. I would
> agree, but the alternative  is not very elegant. I would have to manully
> count the objects in a partition and then move the data and repeat the
> operation if something went wrong.
>
> It is less resource intensive to read a counter value to see if a
> partition needs splitting then to read all the objects in a partition. The
> counter value can be stored in its own table sorting in descending order of
> the counter value.
>
> Does anybody else use another technique for achieving this idempotency
> with counters?
>
> I'm using cassandra 2.0.7.
>
>
>
> Thanks
>
> Jabbar Azam
>
>
>

Re: idempotent counters

Posted by Aaron Morton <aa...@thelastpickle.com>.

> Does anybody else use another technique for achieving this idempotency with counters?

The idempotency problem with counters has to do with what will happen when you get a timeout. If you reply the write there is a chance of the increment been applied twice. This is inherent in the current design. 

Cheers
Aaron 

  
-----------------
Aaron Morton
New Zealand
@aaronmorton

Co-Founder & Principal Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

On 9/05/2014, at 1:07 am, Jabbar Azam <aj...@gmail.com> wrote:

> Hello,
> 
> Do people use counters when they want to have idempotent operations in cassandra?
> 
> I have a use case for using a counter to check for a count of objects in a partition. If the counter is more than some value then the data in the partition is moved into two different partitions. I can't work out how to do this splitting and recover if a problem happens during modification of the counter.
> 
> http://www.ebaytechblog.com/2012/08/14/cassandra-data-modeling-best-practices-part-2 explains that counters shouldn't be used if you want idempotency. I would agree, but the alternative  is not very elegant. I would have to manully count the objects in a partition and then move the data and repeat the operation if something went wrong.
> 
> It is less resource intensive to read a counter value to see if a partition needs splitting then to read all the objects in a partition. The counter value can be stored in its own table sorting in descending order of the counter value.
> 
> Does anybody else use another technique for achieving this idempotency with counters?
> 
> I'm using cassandra 2.0.7.
> 
> 
> 
> Thanks
> 
> Jabbar Azam