Fwd: Cassandra pre 2.1 vs 2.1 counter implementation

[Localhost shell <un...@gmail.com>]

Hey All,


I am new to C* community.

We are planning to use Datastax C* (pre 2.1) in production. We heavily use
counters and it is mostly what we do apart from storing the few months raw
logs in C*.



I have gone through the excellent Sylvain Lesbresne ppt
<http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf_counters.pdf>and
the design doc<https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf>
for
the pre 2.1 counter implementation in detail. Unfortunately the video is
not available.


*Questions:*



1. I know from https://issues.apache.org/jira/browse/CASSANDRA-6504 that
Counters design has changed in 2.1 but not able to get hold of the deign
doc. Can someone plz share the design docs?

    Also according to the 6504 issue and C* codebase CounterMutation.apply(),
C* 2.1 has introduced locks that were not existent before.

How does it impact the write performance of counter as compared to pre 2.1
no lock partition counter implementation?


2.  What were the major concerns (other than idempotency and overcount due
to timeout exceptions) in the pre 2.1 counters architecture that led to a
rewrite of counters implementation?


Thanks for the help.

--Unilocal

Re: Cassandra pre 2.1 vs 2.1 counter implementation

[Robert Coli <rc...@eventbrite.com>]

On Wed, May 21, 2014 at 1:15 PM, Localhost shell <
universal.localhost@gmail.com> wrote:

>
> I am new to C* community.
>
> We are planning to use Datastax C* (pre 2.1) in production. We heavily
> use counters and it is mostly what we do apart from storing the few months
> raw logs in C*.
>

https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/

Most people using Cassandra Counters in production have some combo of low
accuracy requirements, low update rate, or high update rate behind an
in-memory accumulator which is flushed to Cassandra periodically.


> 2.  What were the major concerns (other than idempotency and overcount
> due to timeout exceptions) in the pre 2.1 counters architecture that led
> to a rewrite of counters implementation?
>

https://issues.apache.org/jira/browse/CASSANDRA-4417

And slowness...

https://issues.apache.org/jira/browse/CASSANDRA-4775
and
https://issues.apache.org/jira/browse/CASSANDRA-6504

have a lot of details/background.

=Rob

Re: Cassandra pre 2.1 vs 2.1 counter implementation

[Nate McCall <na...@thelastpickle.com>]

Jonathan covered the changes in some detail at one of our recent meetups
(at about 36 minutes in, give or take):

http://capitalfactory.lifesize.com/videos/video/309/?access_token=shr00000003098845257289498283770596639066969
From:
http://www.meetup.com/Austin-Cassandra-Users/events/158857962/

tl;dr: They will be slower, but accurate.


On Wed, May 21, 2014 at 3:15 PM, Localhost shell <
universal.localhost@gmail.com> wrote:

>
> Hey All,
>
>
> I am new to C* community.
>
> We are planning to use Datastax C* (pre 2.1) in production. We heavily
> use counters and it is mostly what we do apart from storing the few months
> raw logs in C*.
>
>
>
> I have gone through the excellent Sylvain Lesbresne ppt <http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf_counters.pdf>and
> the design doc<https://issues.apache.org/jira/secure/attachment/12459754/Partitionedcountersdesigndoc.pdf> for
> the pre 2.1 counter implementation in detail. Unfortunately the video is
> not available.
>
>
> *Questions:*
>
>
>
> 1. I know from https://issues.apache.org/jira/browse/CASSANDRA-6504 that
> Counters design has changed in 2.1 but not able to get hold of the deign
> doc. Can someone plz share the design docs?
>
>     Also according to the 6504 issue and C* codebase
> CounterMutation.apply(), C* 2.1 has introduced locks that were not
> existent before.
>
> How does it impact the write performance of counter as compared to pre 2.1
> no lock partition counter implementation?
>
>
> 2.  What were the major concerns (other than idempotency and overcount
> due to timeout exceptions) in the pre 2.1 counters architecture that led
> to a rewrite of counters implementation?
>
>
> Thanks for the help.
>
> --Unilocal
>
>


-- 
-----------------
Nate McCall
Austin, TX
@zznate

Co-Founder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com