You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Ananth Gundabattula <ag...@threatmetrix.com> on 2013/06/23 05:25:53 UTC
Upgrade from 1.1.10 to 1.2.4
Hello everybody,
I am trying to perform a rolling upgrade from 1.1.10 to 1.2.4 ( with two patches to 1.2.4 https://issues.apache.org/jira/browse/CASSANDRA-5554
https://issues.apache.org/jira/browse/CASSANDRA-5418 as they might effect us in production )
I was wondering if anyone was able to perform a successful rolling upgrade from 1.1.10 to 1.2.4?
I tried both a rolling upgrade while other nodes are on 1.1.10 version and also while all nodes in the cluster were shutdown and just the new version cassandra node coming up. The 1.1.10 version nodes see the 1.2.4 version node up but the 1.2.4 version node crashes a few seconds after the start up.
I see the following exception in the logs when the 1.2.4 node starts up.
……
INFO 03:03:26,399 Log replay complete, 13 replayed mutations
INFO 03:03:26,631 Cassandra version: 1.2.4-SNAPSHOT
INFO 03:03:26,631 Thrift API version: 19.35.0
INFO 03:03:26,632 CQL supported versions: 2.0.0,3.0.1 (default: 3.0.1)
INFO 03:03:26,660 Starting up server gossip
INFO 03:03:26,671 Enqueuing flush of Memtable-local@1284117703(253/253 serialized/live bytes, 9 ops)
INFO 03:03:26,672 Writing Memtable-local@1284117703(253/253 serialized/live bytes, 9 ops)
INFO 03:03:26,676 Completed flushing /data/cassandra/data/system/local/system-local-ib-4-Data.db (250 bytes) for commitlog position ReplayPosition(segmentId=1371956606055, position=50387)
INFO 03:03:26,684 Compacting [SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-3-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-2-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-4-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-1-Data.db')]
INFO 03:03:26,706 Compacted 4 sstables to [/data/cassandra/data/system/local/system-local-ib-5,]. 852 bytes to 457 (~53% of original) in 19ms = 0.022938MB/s. 4 total rows, 1 unique. Row merge counts were {1:0, 2:0, 3:0, 4:1, }
INFO 03:03:26,769 Starting Messaging Service on port 7000
ERROR 03:03:26,842 Exception encountered during startup
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
Exception encountered during startup: null
ERROR 03:03:26,848 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:507)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.lang.Thread.run(Thread.java:722)
Regards,
Ananth
Re: Upgrade from 1.1.10 to 1.2.4
Posted by Ananth Gundabattula <ag...@threatmetrix.com>.
Hello Rob,
I ran into the stack trace when the situation was :
num_tokens unset ( by this I mean not specifying anything ) and
initial_token set to some value.
I was initially under the impression that specifying num_tokens will over
ride the initial_token value and hence left num_tokens blank. I was able
to get past that exception only when num_token was specified with a value
of 1.
Regards,
Ananth
On 6/25/13 3:27 AM, "Robert Coli" <rc...@eventbrite.com> wrote:
>On Sun, Jun 23, 2013 at 2:31 AM, Ananth Gundabattula
><ag...@threatmetrix.com> wrote:
>> Looks like the cause of the error was because of not specifying
>>num_tokens
>> in the cassandra.yaml file. I was under the impression that setting a
>>value
>> of num_tokens will override the initial_token value . Looks like we
>>need to
>> set num_tokens to 1 to get around this error. Not specifying anything
>>causes
>> the above error.
>
>My understanding is that the 1.2.x behavior here is :
>
>1) initial_token set, num_tokens set = cassandra picks the num_tokens
>value, ignores initial_token
>2) initial_token unset, num_tokens unset = cassandra (until 2.0) picks
>a single token via range bisection
>3) initial_token unset, num_tokens set = cassandra uses num_tokens
>number of vnodes
>
>Are you saying this is not the behavior you saw?
>
>=Rob
Re: Upgrade from 1.1.10 to 1.2.4
Posted by Robert Coli <rc...@eventbrite.com>.
On Sun, Jun 23, 2013 at 2:31 AM, Ananth Gundabattula
<ag...@threatmetrix.com> wrote:
> Looks like the cause of the error was because of not specifying num_tokens
> in the cassandra.yaml file. I was under the impression that setting a value
> of num_tokens will override the initial_token value . Looks like we need to
> set num_tokens to 1 to get around this error. Not specifying anything causes
> the above error.
My understanding is that the 1.2.x behavior here is :
1) initial_token set, num_tokens set = cassandra picks the num_tokens
value, ignores initial_token
2) initial_token unset, num_tokens unset = cassandra (until 2.0) picks
a single token via range bisection
3) initial_token unset, num_tokens set = cassandra uses num_tokens
number of vnodes
Are you saying this is not the behavior you saw?
=Rob
Re: Upgrade from 1.1.10 to 1.2.4
Posted by Ananth Gundabattula <ag...@threatmetrix.com>.
Looks like the cause of the error was because of not specifying num_tokens in the cassandra.yaml file. I was under the impression that setting a value of num_tokens will override the initial_token value . Looks like we need to set num_tokens to 1 to get around this error. Not specifying anything causes the above error.
Regards,
Ananth
From: Ananth Gundabattula <ag...@threatmetrix.com>>
Reply-To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Date: Sunday, June 23, 2013 1:25 PM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Upgrade from 1.1.10 to 1.2.4
Hello everybody,
I am trying to perform a rolling upgrade from 1.1.10 to 1.2.4 ( with two patches to 1.2.4 https://issues.apache.org/jira/browse/CASSANDRA-5554
https://issues.apache.org/jira/browse/CASSANDRA-5418 as they might effect us in production )
I was wondering if anyone was able to perform a successful rolling upgrade from 1.1.10 to 1.2.4?
I tried both a rolling upgrade while other nodes are on 1.1.10 version and also while all nodes in the cluster were shutdown and just the new version cassandra node coming up. The 1.1.10 version nodes see the 1.2.4 version node up but the 1.2.4 version node crashes a few seconds after the start up.
I see the following exception in the logs when the 1.2.4 node starts up.
……
INFO 03:03:26,399 Log replay complete, 13 replayed mutations
INFO 03:03:26,631 Cassandra version: 1.2.4-SNAPSHOT
INFO 03:03:26,631 Thrift API version: 19.35.0
INFO 03:03:26,632 CQL supported versions: 2.0.0,3.0.1 (default: 3.0.1)
INFO 03:03:26,660 Starting up server gossip
INFO 03:03:26,671 Enqueuing flush of Memtable-local@1284117703(253/253 serialized/live bytes, 9 ops)
INFO 03:03:26,672 Writing Memtable-local@1284117703(253/253 serialized/live bytes, 9 ops)
INFO 03:03:26,676 Completed flushing /data/cassandra/data/system/local/system-local-ib-4-Data.db (250 bytes) for commitlog position ReplayPosition(segmentId=1371956606055, position=50387)
INFO 03:03:26,684 Compacting [SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-3-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-2-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-4-Data.db'), SSTableReader(path='/data/cassandra/data/system/local/system-local-ib-1-Data.db')]
INFO 03:03:26,706 Compacted 4 sstables to [/data/cassandra/data/system/local/system-local-ib-5,]. 852 bytes to 457 (~53% of original) in 19ms = 0.022938MB/s. 4 total rows, 1 unique. Row merge counts were {1:0, 2:0, 3:0, 4:1, }
INFO 03:03:26,769 Starting Messaging Service on port 7000
ERROR 03:03:26,842 Exception encountered during startup
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:716)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
Exception encountered during startup: null
ERROR 03:03:26,848 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:507)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.lang.Thread.run(Thread.java:722)
Regards,
Ananth