Re: Implementing Counter on Cassandra

[Benjamin Black <b...@b3k.us>]

ZK is way overkill for counters.  memcache and redis are much better at the job.

On Tue, Jun 29, 2010 at 12:32 PM, Jonathan Shook <js...@gmail.com> wrote:
> Until then, a pragmatic solution, however undesirable, would be to
> only have a single logical thread/task/actor that is allowed to
> read,modify,update. If this doesn't work for your application, then a
> (distributed) lock manager may be used until such time that you can
> take it out. Some are using ZooKeeper for this.
>
>
> On Tue, Jun 29, 2010 at 11:45 AM, Ryan King <ry...@twitter.com> wrote:
>> On Tue, Jun 29, 2010 at 9:42 AM, Utku Can Topçu <ut...@topcu.gen.tr> wrote:
>>> Hey Guys,
>>>
>>> Currently in a project I'm involved in, I need to have some columns holding
>>> incremented data.
>>> The easy approach for implementing a counter with increments is right now as
>>> I figured out is "read -> increment -> insert" however this approach is not
>>> an atomic operation and can easily be corrupted in time.
>>>
>>> Do you have any best practices in implementing an atomic counter on
>>> Cassandra?
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-1072
>>
>

Re: Implementing Counter on Cassandra

[Shahan Khan <co...@shahan.me>]

Hi Utku,

If I'm not mistaken, I think for this case redis would be a good use case
for keeping counters. The actual data is (I believe) still being stored In
Cassandra. The data could be copied out of redis back into Cassandra every
night / hour / minute depending on the users need, and removed from redis
as the need for old data to be incremented becomes unnecessary.    

Also, an article on Gooogle App Engine recommends sharding Counters for
better performance, though this may not be needed as redis supports a INCR
command. Article:
http://code.google.com/appengine/articles/sharding_counters.html

Of course, as you mentioned, the drawback is the lack of range queries in
memcache and redis.

Shahan

On Thu, 1 Jul 2010 10:08:49 +0200, Utku Can Topçu  wrote:  

Hi Benjamin, as far as I know both memcache and redis does not support
range queries on keys. So it would be really hard to update the cassandra
columns reading from these and then updating them to cassandra.

 On Thu, Jul 1, 2010 at 3:57 AM, Benjamin Black  wrote:
 ZK is way overkill for counters. memcache and redis are much better at
the job.

 On Tue, Jun 29, 2010 at 12:32 PM, Jonathan Shook  wrote:
 > Until then, a pragmatic solution, however undesirable, would be to
 > only have a single logical thread/task/actor that is allowed to
 > read,modify,update. If this doesn't work for your application, then a
 > (distributed) lock manager may be used until such time that you can
 > take it out. Some are using ZooKeeper for this.
 >
 >
 > On Tue, Jun 29, 2010 at 11:45 AM, Ryan King  wrote:
 >> On Tue, Jun 29, 2010 at 9:42 AM, Utku Can Topçu  wrote:
 >>> Hey Guys,
 >>>
 >>> Currently in a project I'm involved in, I need to have some columns
holding
 >>> incremented data.
 >>> The easy approach for implementing a counter with increments is right
now as
 >>> I figured out is "read -> increment -> insert" however this approach
is not
 >>> an atomic operation and can easily be corrupted in time.
 >>>
 >>> Do you have any best practices in implementing an atomic counter on
 >>> Cassandra?
 >>
 >> https://issues.apache.org/jira/browse/CASSANDRA-1072 [5]
 >>
 >    

 

Links:
------
[1] mailto:b@b3k.us
[2] mailto:jshook@gmail.com
[3] mailto:ryan@twitter.com
[4] mailto:utku@topcu.gen.tr
[5] https://issues.apache.org/jira/browse/CASSANDRA-1072

Re: Implementing Counter on Cassandra

[Utku Can Topçu <ut...@topcu.gen.tr>]

Hi Benjamin, as far as I know both memcache and redis does not support range
queries on keys. So it would be really hard to update the cassandra columns
reading from these and then updating them to cassandra.

On Thu, Jul 1, 2010 at 3:57 AM, Benjamin Black <b...@b3k.us> wrote:

> ZK is way overkill for counters.  memcache and redis are much better at the
> job.
>
> On Tue, Jun 29, 2010 at 12:32 PM, Jonathan Shook <js...@gmail.com> wrote:
> > Until then, a pragmatic solution, however undesirable, would be to
> > only have a single logical thread/task/actor that is allowed to
> > read,modify,update. If this doesn't work for your application, then a
> > (distributed) lock manager may be used until such time that you can
> > take it out. Some are using ZooKeeper for this.
> >
> >
> > On Tue, Jun 29, 2010 at 11:45 AM, Ryan King <ry...@twitter.com> wrote:
> >> On Tue, Jun 29, 2010 at 9:42 AM, Utku Can Topçu <ut...@topcu.gen.tr>
> wrote:
> >>> Hey Guys,
> >>>
> >>> Currently in a project I'm involved in, I need to have some columns
> holding
> >>> incremented data.
> >>> The easy approach for implementing a counter with increments is right
> now as
> >>> I figured out is "read -> increment -> insert" however this approach is
> not
> >>> an atomic operation and can easily be corrupted in time.
> >>>
> >>> Do you have any best practices in implementing an atomic counter on
> >>> Cassandra?
> >>
> >> https://issues.apache.org/jira/browse/CASSANDRA-1072
> >>
> >
>