You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Martin Lansler <ma...@gmail.com> on 2011/08/11 16:30:11 UTC
Best practices when deploying & upgrading a cassandra cluster
(Note: This is a repost from another thread which did not have a
relevant subject, sorry for the spamming)
Hi Eldad / All,
On Wed, Aug 10, 2011 at 8:32 AM, Eldad Yamin <el...@gmail.com> wrote:
> Can you please explain how did you upgraded. something like step-by-step.
> Thanks!
I took the liberty of replying to the group as it would be interesting
to hear how other folks out there are doing it...
I'm *not* running a prod system, just a test system of three nodes on
my laptop. So it would be nice to hear about real setups. Here is my
test setup:
apache-cassandra -> apache-cassandra-0.8.3
apache-cassandra-0.8.2/
apache-cassandra-0.8.3/
node1/
node2/
node3/
All nodeX look like:
bin -> ../apache-cassandra/bin/
commitlog/
conf/
data/
interface -> ../apache-cassandra/interface/
lib -> ../apache-cassandra/lib/
saved_caches/
The 'conf' directory is copied into each node from the virgin
cassandra distribution. I then create a local GIT repo and add the
'conf' directory so I can track any configuration changes on a node.
Then relevant node specific configuration settings are set. The
'commitlog', 'data' and 'saved_caches' are created by cassandra and
must be configured in 'cassandra.yaml' for each node.
When I upgrade I do the following:
1.
Make a diff of the new conf files from the new version so that get
new parameters etc... I use emacs ediff-mode.
2.
Remove the old "apache-cassandra" symlink and point it to the new cassandra dist
3.
In a rolling fashion stop one node, and then restart it... as the
symlink is changes it will then boot with the upgraded cassandra dist.
(remember to cd out & in of the bin/ dir otherwise you will still be
in the old directory).
(4).
Should something break... just re-create the old symlink and restart
the node (provided cassandra has not performed any non backwards
compatible changes to the db files, should be noted in the README)
That's pretty much it.
On a prod setup one would probably use a tool such as puppet
(www.puppetlabs.com/) to ease setting up on many nodes... But there
are many ways to do this, for instance pssh
(http://code.google.com/p/parallel-ssh/).
Regards,
-Martin
Re: Best practices when deploying & upgrading a cassandra cluster
Posted by aaron morton <aa...@thelastpickle.com>.
There can be bugs in the trunk. Stuff can be put in and the taken out.
The tagged and packages releases are considered stable and supported. You can run trunk in prod if you want to, it's good to get tested, but you will need to pay attention to whats happening with the code base and be prepared to accept any consequences.
You will have a less exciting life if you use the official releases.
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 15/08/2011, at 6:07 AM, Eldad Yamin wrote:
>
> Is there any good reason why shouldn't we build the latest version from source?
>
> Thanks!
> On Fri, Aug 12, 2011 at 12:18 AM, aaron morton <aa...@thelastpickle.com> wrote:
> In a non dev system it's a lot easier to use the packages
> http://wiki.apache.org/cassandra/DebianPackaging
> http://www.datastax.com/docs/0.8/install/packaged_releases
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 12 Aug 2011, at 02:30, Martin Lansler wrote:
>
>> (Note: This is a repost from another thread which did not have a
>> relevant subject, sorry for the spamming)
>>
>> Hi Eldad / All,
>>
>> On Wed, Aug 10, 2011 at 8:32 AM, Eldad Yamin <el...@gmail.com> wrote:
>>> Can you please explain how did you upgraded. something like step-by-step.
>>> Thanks!
>>
>> I took the liberty of replying to the group as it would be interesting
>> to hear how other folks out there are doing it...
>>
>> I'm *not* running a prod system, just a test system of three nodes on
>> my laptop. So it would be nice to hear about real setups. Here is my
>> test setup:
>>
>> apache-cassandra -> apache-cassandra-0.8.3
>> apache-cassandra-0.8.2/
>> apache-cassandra-0.8.3/
>> node1/
>> node2/
>> node3/
>>
>> All nodeX look like:
>> bin -> ../apache-cassandra/bin/
>> commitlog/
>> conf/
>> data/
>> interface -> ../apache-cassandra/interface/
>> lib -> ../apache-cassandra/lib/
>> saved_caches/
>>
>> The 'conf' directory is copied into each node from the virgin
>> cassandra distribution. I then create a local GIT repo and add the
>> 'conf' directory so I can track any configuration changes on a node.
>> Then relevant node specific configuration settings are set. The
>> 'commitlog', 'data' and 'saved_caches' are created by cassandra and
>> must be configured in 'cassandra.yaml' for each node.
>>
>> When I upgrade I do the following:
>>
>> 1.
>> Make a diff of the new conf files from the new version so that get
>> new parameters etc... I use emacs ediff-mode.
>> 2.
>> Remove the old "apache-cassandra" symlink and point it to the new cassandra dist
>> 3.
>> In a rolling fashion stop one node, and then restart it... as the
>> symlink is changes it will then boot with the upgraded cassandra dist.
>> (remember to cd out & in of the bin/ dir otherwise you will still be
>> in the old directory).
>> (4).
>> Should something break... just re-create the old symlink and restart
>> the node (provided cassandra has not performed any non backwards
>> compatible changes to the db files, should be noted in the README)
>>
>> That's pretty much it.
>>
>> On a prod setup one would probably use a tool such as puppet
>> (www.puppetlabs.com/) to ease setting up on many nodes... But there
>> are many ways to do this, for instance pssh
>> (http://code.google.com/p/parallel-ssh/).
>>
>> Regards,
>> -Martin
>
>
Re: Best practices when deploying & upgrading a cassandra cluster
Posted by Eldad Yamin <el...@gmail.com>.
Is there any good reason why shouldn't we build the latest version from
source?
Thanks!
On Fri, Aug 12, 2011 at 12:18 AM, aaron morton <aa...@thelastpickle.com>wrote:
> In a non dev system it's a lot easier to use the packages
> http://wiki.apache.org/cassandra/DebianPackaging
> http://www.datastax.com/docs/0.8/install/packaged_releases
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 12 Aug 2011, at 02:30, Martin Lansler wrote:
>
> (Note: This is a repost from another thread which did not have a
> relevant subject, sorry for the spamming)
>
> Hi Eldad / All,
>
> On Wed, Aug 10, 2011 at 8:32 AM, Eldad Yamin <el...@gmail.com> wrote:
>
> Can you please explain how did you upgraded. something like step-by-step.
>
> Thanks!
>
>
> I took the liberty of replying to the group as it would be interesting
> to hear how other folks out there are doing it...
>
> I'm *not* running a prod system, just a test system of three nodes on
> my laptop. So it would be nice to hear about real setups. Here is my
> test setup:
>
> apache-cassandra -> apache-cassandra-0.8.3
> apache-cassandra-0.8.2/
> apache-cassandra-0.8.3/
> node1/
> node2/
> node3/
>
> All nodeX look like:
> bin -> ../apache-cassandra/bin/
> commitlog/
> conf/
> data/
> interface -> ../apache-cassandra/interface/
> lib -> ../apache-cassandra/lib/
> saved_caches/
>
> The 'conf' directory is copied into each node from the virgin
> cassandra distribution. I then create a local GIT repo and add the
> 'conf' directory so I can track any configuration changes on a node.
> Then relevant node specific configuration settings are set. The
> 'commitlog', 'data' and 'saved_caches' are created by cassandra and
> must be configured in 'cassandra.yaml' for each node.
>
> When I upgrade I do the following:
>
> 1.
> Make a diff of the new conf files from the new version so that get
> new parameters etc... I use emacs ediff-mode.
> 2.
> Remove the old "apache-cassandra" symlink and point it to the new cassandra
> dist
> 3.
> In a rolling fashion stop one node, and then restart it... as the
> symlink is changes it will then boot with the upgraded cassandra dist.
> (remember to cd out & in of the bin/ dir otherwise you will still be
> in the old directory).
> (4).
> Should something break... just re-create the old symlink and restart
> the node (provided cassandra has not performed any non backwards
> compatible changes to the db files, should be noted in the README)
>
> That's pretty much it.
>
> On a prod setup one would probably use a tool such as puppet
> (www.puppetlabs.com/) to ease setting up on many nodes... But there
> are many ways to do this, for instance pssh
> (http://code.google.com/p/parallel-ssh/).
>
> Regards,
> -Martin
>
>
>
Re: Best practices when deploying & upgrading a cassandra cluster
Posted by aaron morton <aa...@thelastpickle.com>.
In a non dev system it's a lot easier to use the packages
http://wiki.apache.org/cassandra/DebianPackaging
http://www.datastax.com/docs/0.8/install/packaged_releases
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 12 Aug 2011, at 02:30, Martin Lansler wrote:
> (Note: This is a repost from another thread which did not have a
> relevant subject, sorry for the spamming)
>
> Hi Eldad / All,
>
> On Wed, Aug 10, 2011 at 8:32 AM, Eldad Yamin <el...@gmail.com> wrote:
>> Can you please explain how did you upgraded. something like step-by-step.
>> Thanks!
>
> I took the liberty of replying to the group as it would be interesting
> to hear how other folks out there are doing it...
>
> I'm *not* running a prod system, just a test system of three nodes on
> my laptop. So it would be nice to hear about real setups. Here is my
> test setup:
>
> apache-cassandra -> apache-cassandra-0.8.3
> apache-cassandra-0.8.2/
> apache-cassandra-0.8.3/
> node1/
> node2/
> node3/
>
> All nodeX look like:
> bin -> ../apache-cassandra/bin/
> commitlog/
> conf/
> data/
> interface -> ../apache-cassandra/interface/
> lib -> ../apache-cassandra/lib/
> saved_caches/
>
> The 'conf' directory is copied into each node from the virgin
> cassandra distribution. I then create a local GIT repo and add the
> 'conf' directory so I can track any configuration changes on a node.
> Then relevant node specific configuration settings are set. The
> 'commitlog', 'data' and 'saved_caches' are created by cassandra and
> must be configured in 'cassandra.yaml' for each node.
>
> When I upgrade I do the following:
>
> 1.
> Make a diff of the new conf files from the new version so that get
> new parameters etc... I use emacs ediff-mode.
> 2.
> Remove the old "apache-cassandra" symlink and point it to the new cassandra dist
> 3.
> In a rolling fashion stop one node, and then restart it... as the
> symlink is changes it will then boot with the upgraded cassandra dist.
> (remember to cd out & in of the bin/ dir otherwise you will still be
> in the old directory).
> (4).
> Should something break... just re-create the old symlink and restart
> the node (provided cassandra has not performed any non backwards
> compatible changes to the db files, should be noted in the README)
>
> That's pretty much it.
>
> On a prod setup one would probably use a tool such as puppet
> (www.puppetlabs.com/) to ease setting up on many nodes... But there
> are many ways to do this, for instance pssh
> (http://code.google.com/p/parallel-ssh/).
>
> Regards,
> -Martin