changing several things (almost) at once; is this the right order to make the changes?

[Brian Tarbox <ta...@cabotresearch.com>]

We're making several changes and I'd to confirm that our order of making
them is reasonable.  Right now we have 4 node system at replicationFactor=2
running 1.1.6.

We've moving to a 6 node system at rf=3 running 1.2.12 (I guess).

We think the order should be:
1) change to rf=3 and run repair on all nodes while still at 1.1.6
2) upgrade to 1.1.10 (latest on that branch?)
3) upgrade to 1.2.12 (latest on that branch?)
4) run the convert-to-v_Node command
5) add two more servers

Is that reasonable?  Thanks.  We run in ec2 and I'm planning on testing it
all on a new set of servers just in case but figured I'd ask the experts
first in case I'm doing something foolish.

Thanks,

Brian Tarbox

Re: changing several things (almost) at once; is this the right order to make the changes?

[Robert Coli <rc...@eventbrite.com>]

On Mon, Dec 2, 2013 at 1:08 PM, Brian Tarbox <ta...@cabotresearch.com>wrote:

> We're making several changes and I'd to confirm that our order of making
> them is reasonable.  Right now we have 4 node system at replicationFactor=2
> running 1.1.6.
>
> We've moving to a 6 node system at rf=3 running 1.2.12 (I guess).
>
> We think the order should be:
> 1) change to rf=3 and run repair on all nodes while still at 1.1.6
>

Yes, being aware that you will get false "no data" reads from the third
replica at CL.ONE until your repair completes.


> 2) upgrade to 1.1.10 (latest on that branch?)
>

Unless NEWS.txt specifies that you need to do this, you can probably skip
it. From memory, I believe you can skip it.


> 3) upgrade to 1.2.12 (latest on that branch?)
>

Yes.


>  4) run the convert-to-v_Node command
>

If you mean shuffle, I feel bound to tell you that no one has successfully
run shuffle on an actual production cluster[1]. I conjecture that you are
in production because you are running 1.1.x.

You might be the first to successfully run shuffle in production, but you
probably do not want to try to be?


> 5) add two more servers
>

If you're going to add servers anyway, you might want to do the "new
datacenter(s)" process for upgrading to Vnodes.

=Rob
[1] rbranson apparently did a shuffle-like activity successfully, but by
adding two additional DCs, one with a node with enough disk space to hold
the entire cluster's data...