incrementally acquiring tokens

[Jonathan Haddad <jo...@jonhaddad.com>]

I've been thinking a bit about the problem of fat nodes (say, 20gb per
node).  My understanding is that the amount of overhead of adding a new
node into the ring is massive with fat nodes due to the fact that you have
to stream in 20TB which takes forever.

In a scenario where a given node only has a single token, my assumption is
this forces Cassandra to send the entire token range to the new node.

As a potential approach, would it be possible for a node to incrementally
acquire tokens, and as a result incrementally stream?  You could have a
node serving requests after acquiring 1 token, and it would gradually take
ownership of more and more of the ring as it bootstraps.

Thoughts?
Jon

Re: incrementally acquiring tokens

[Tyler Hobbs <ty...@datastax.com>]

On Sat, Dec 13, 2014 at 11:33 AM, Jonathan Haddad <jo...@jonhaddad.com> wrote:

>
> As a potential approach, would it be possible for a node to incrementally
> acquire tokens, and as a result incrementally stream?  You could have a
> node serving requests after acquiring 1 token, and it would gradually take
> ownership of more and more of the ring as it bootstraps.


Yes, it's definitely theoretically possible.  The main hurdle is adding
support for "bootstrapping" a single token to gossip.  Gossips is
unfortunately fairly inflexible, so this would probably be a little tricky,
and would mostly likely have to target 3.1.


-- 
Tyler Hobbs
DataStax <http://datastax.com/>