Consistent counters (was: Memcached protocol)

[Paul Prescod <pr...@gmail.com>]

I *believe* that the key messages of those blog posts was:

 1. Using distributed vector clocks are easy once they are implemented.

 2. Implementing distributed vector clocks is hard on the datastore vendor.

 3. If you have long-term network partitions you're kind of screwed (which
is probably true in any case)

Do you agree that those are the key points?

Or to put it another way: if you are willing to give up one of availability
or partition-tolerance, then do you agree that 100% "eventually consistent
counters" are possible?

And if so, is it your plan that "eventually consistent counters" be possible
in a future version of Cassandra?

On Tue, Apr 6, 2010 at 8:52 AM, Jonathan Ellis <jb...@gmail.com> wrote:

> On Mon, Apr 5, 2010 at 6:48 PM, Tatu Saloranta <ts...@gmail.com>
> wrote:
> > I would think that there is also possibility of losing some
> > increments, or perhaps getting duplicate increments?
> > It is not just isolation but also correctness that is hard to maintain
> > but correctness also. This can be more easily worked around in cases
> > where there is additional data that can be used to resolve potentially
> > ambiguous changes (like inferring which of shopping cart additions are
> > real, which duplicates).
> > With more work I am sure it is possible to get things mostly working,
> > it's just question of cost/benefit for specific use cases.
>
> Let me inject a couple useful references:
>
> http://pl.atyp.us/wordpress/?p=2601
> http://blog.basho.com/2010/04/05/why-vector-clocks-are-hard/
>

Re: Consistent counters (was: Memcached protocol)

[Jonathan Ellis <jb...@gmail.com>]

On Tue, Apr 6, 2010 at 2:27 PM, Paul Prescod <pr...@gmail.com> wrote:
> I *believe* that the key messages of those blog posts was:
>  1. Using distributed vector clocks are easy once they are implemented.
>  2. Implementing distributed vector clocks is hard on the datastore vendor.
>  3. If you have long-term network partitions you're kind of screwed (which
> is probably true in any case)

I think this is not a problem in practice, where "long term" means
hours not weeks.

> And if so, is it your plan that "eventually consistent counters" be possible
> in a future version of Cassandra?

Yes, that is the main reason Digg is working on 580.

-Jonathan