You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Christoph Eberhardt <ch...@googlemail.com> on 2012/05/16 14:12:59 UTC

Startup fails after updgrading from 1.0.8 to 1.1.0

Hi there,

if updgraded cassandra from 1.0.8 to 1.1.0. It seemed to work in the first place, all seemed to work fine. So I started upgrading the rest of the cluster (at the time only one other node, that is a replica). After having a several errors, I restarted the cluster and now cassandra won't even start up. Startup fails with the following error message: 


INFO 13:59:05,175 Logging initialized INFO 13:59:05,178 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.6.0_26 INFO 13:59:05,178 Heap size: 8420720640/8420720640 INFO 13:59:05,178 Classpath: bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.1.0.jar:bin/../lib/apache-cassandra-clientutil-1.1.0.jar:bin/../lib/apache-cassandra-thrift-1.1.0.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.9.2.jar:bin/../lib/jackson-mapper-asl-1.9.2.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.7.0.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/metrics-core-2.0.3.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/snaptree-0.1.jar:bin/../lib/jamm-0.2.5.jar INFO 13:59:07,312 JNA mlockall successful INFO 13:59:07,325 Loading settings from file:/opt/cassandra/apache-cassandra-1.1.0/conf/cassandra.yaml INFO 13:59:07,419 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 13:59:07,567 Global memtable threshold is enabled at 2676MB
 INFO 13:59:07,654 Initializing key cache with capacity of 100 MBs.
 INFO 13:59:07,661 Scheduling key cache save to each 14400 seconds (going to save all keys).
 INFO 13:59:07,662 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider
 INFO 13:59:07,664 Scheduling row cache save to each 0 seconds (going to save all keys).
 INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-9 (28520 bytes)
 INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-10 (28520 bytes)
 INFO 13:59:07,746 Opening /opt/cassandra/database/data/system/NodeIdInfo/system-NodeIdInfo-hc-1 (187 bytes)
 INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-3 (1892 bytes)
 INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-1 (1892 bytes)
 INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-2 (1892 bytes)
 INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-34 (247 bytes)
 INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-35 (247 bytes)
 INFO 13:59:07,815 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-32 (490 bytes)
 INFO 13:59:07,816 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-33 (115 bytes)
 INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-9 (506 bytes)
 INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-10 (506 bytes)
 INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-74 (148 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-72 (406 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-73 (80 bytes)ERROR 13:59:08,244 Exception encountered during startup
java.lang.IllegalArgumentException: value already present: 1034
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
        at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
        at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
        at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
        at org.apache.cassandra.config.Schema.load(Schema.java:385)
        at org.apache.cassandra.config.Schema.load(Schema.java:106)
        at org.apache.cassandra.config.Schema.load(Schema.java:91)
        at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
java.lang.IllegalArgumentException: value already present: 1034
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
        at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
        at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
        at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
        at org.apache.cassandra.config.Schema.load(Schema.java:385)
        at org.apache.cassandra.config.Schema.load(Schema.java:106)
        at org.apache.cassandra.config.Schema.load(Schema.java:91)
        at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
Exception encountered during startup: value already present: 1034


What can I do to get my cluster back online?

Thank you

Christoph Eberhardt

Re: Startup fails after updgrading from 1.0.8 to 1.1.0

Posted by Dave Brosius <db...@mebigfatguy.com>.

tracking issue here: https://issues.apache.org/jira/browse/CASSANDRA-4251
might be related to: https://issues.apache.org/jira/browse/CASSANDRA-3794

On 05/16/2012 08:12 AM, Christoph Eberhardt wrote:
> Hi there,
>
> if updgraded cassandra from 1.0.8 to 1.1.0. It seemed to work in the first place, all seemed to work fine. So I started upgrading the rest of the cluster (at the time only one other node, that is a replica). After having a several errors, I restarted the cluster and now cassandra won't even start up. Startup fails with the following error message:
>
>
> INFO 13:59:05,175 Logging initialized INFO 13:59:05,178 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.6.0_26 INFO 13:59:05,178 Heap size: 8420720640/8420720640 INFO 13:59:05,178 Classpath: bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.1.0.jar:bin/../lib/apache-cassandra-clientutil-1.1.0.jar:bin/../lib/apache-cassandra-thrift-1.1.0.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.9.2.jar:bin/../lib/jackson-mapper-asl-1.9.2.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.7.0.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/metrics-core-2.0.3.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/snaptree-0.1.jar:bin/../lib/jamm-0.2.5.jar INFO 13:59:07,312 JNA mlockall successful INFO 13:59:07,325 Loading settings from file:/opt/cassandra/apache-cassandra-1.1.0/conf/cassandra.yaml INFO 13:59:07,419 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 13:59:07,567 Global memtable threshold is enabled at 2676MB
>   INFO 13:59:07,654 Initializing key cache with capacity of 100 MBs.
>   INFO 13:59:07,661 Scheduling key cache save to each 14400 seconds (going to save all keys).
>   INFO 13:59:07,662 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider
>   INFO 13:59:07,664 Scheduling row cache save to each 0 seconds (going to save all keys).
>   INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-9 (28520 bytes)
>   INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-10 (28520 bytes)
>   INFO 13:59:07,746 Opening /opt/cassandra/database/data/system/NodeIdInfo/system-NodeIdInfo-hc-1 (187 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-3 (1892 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-1 (1892 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-2 (1892 bytes)
>   INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-34 (247 bytes)
>   INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-35 (247 bytes)
>   INFO 13:59:07,815 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-32 (490 bytes)
>   INFO 13:59:07,816 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-33 (115 bytes)
>   INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-9 (506 bytes)
>   INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-10 (506 bytes)
>   INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-74 (148 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-72 (406 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-73 (80 bytes)ERROR 13:59:08,244 Exception encountered during startup
> java.lang.IllegalArgumentException: value already present: 1034
>          at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
>          at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
>          at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
>          at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
>          at org.apache.cassandra.config.Schema.load(Schema.java:385)
>          at org.apache.cassandra.config.Schema.load(Schema.java:106)
>          at org.apache.cassandra.config.Schema.load(Schema.java:91)
>          at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
>          at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> java.lang.IllegalArgumentException: value already present: 1034
>          at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
>          at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
>          at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
>          at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
>          at org.apache.cassandra.config.Schema.load(Schema.java:385)
>          at org.apache.cassandra.config.Schema.load(Schema.java:106)
>          at org.apache.cassandra.config.Schema.load(Schema.java:91)
>          at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
>          at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Exception encountered during startup: value already present: 1034
>
>
> What can I do to get my cluster back online?
>
> Thank you
>
> Christoph Eberhardt
>
>
>
>

Re: Startup fails after updgrading from 1.0.8 to 1.1.0

Posted by Dave Brosius <db...@mebigfatguy.com>.

Might be related to

https://issues.apache.org/jira/browse/CASSANDRA-3794



On 05/16/2012 08:12 AM, Christoph Eberhardt wrote:
> Hi there,
>
> if updgraded cassandra from 1.0.8 to 1.1.0. It seemed to work in the first place, all seemed to work fine. So I started upgrading the rest of the cluster (at the time only one other node, that is a replica). After having a several errors, I restarted the cluster and now cassandra won't even start up. Startup fails with the following error message:
>
>
> INFO 13:59:05,175 Logging initialized INFO 13:59:05,178 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.6.0_26 INFO 13:59:05,178 Heap size: 8420720640/8420720640 INFO 13:59:05,178 Classpath: bin/../conf:bin/../build/classes/main:bin/../build/classes/thrift:bin/../lib/antlr-3.2.jar:bin/../lib/apache-cassandra-1.1.0.jar:bin/../lib/apache-cassandra-clientutil-1.1.0.jar:bin/../lib/apache-cassandra-thrift-1.1.0.jar:bin/../lib/avro-1.4.0-fixes.jar:bin/../lib/avro-1.4.0-sources-fixes.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/compress-lzf-0.8.4.jar:bin/../lib/concurrentlinkedhashmap-lru-1.2.jar:bin/../lib/guava-r08.jar:bin/../lib/high-scale-lib-1.1.2.jar:bin/../lib/jackson-core-asl-1.9.2.jar:bin/../lib/jackson-mapper-asl-1.9.2.jar:bin/../lib/jamm-0.2.5.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-0.7.0.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/metrics-core-2.0.3.jar:bin/../lib/mx4j-tools.jar:bin/../lib/servlet-api-2.5-20081211.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.1.jar:bin/../lib/snakeyaml-1.6.jar:bin/../lib/snappy-java-1.0.4.1.jar:bin/../lib/snaptree-0.1.jar:bin/../lib/jamm-0.2.5.jar INFO 13:59:07,312 JNA mlockall successful INFO 13:59:07,325 Loading settings from file:/opt/cassandra/apache-cassandra-1.1.0/conf/cassandra.yaml INFO 13:59:07,419 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap INFO 13:59:07,567 Global memtable threshold is enabled at 2676MB
>   INFO 13:59:07,654 Initializing key cache with capacity of 100 MBs.
>   INFO 13:59:07,661 Scheduling key cache save to each 14400 seconds (going to save all keys).
>   INFO 13:59:07,662 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider
>   INFO 13:59:07,664 Scheduling row cache save to each 0 seconds (going to save all keys).
>   INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-9 (28520 bytes)
>   INFO 13:59:07,717 Opening /opt/cassandra/database/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-10 (28520 bytes)
>   INFO 13:59:07,746 Opening /opt/cassandra/database/data/system/NodeIdInfo/system-NodeIdInfo-hc-1 (187 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-3 (1892 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-1 (1892 bytes)
>   INFO 13:59:07,777 Opening /opt/cassandra/database/data/system/schema_columns/system-schema_columns-hc-2 (1892 bytes)
>   INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-34 (247 bytes)
>   INFO 13:59:07,790 Opening /opt/cassandra/database/data/system/Versions/system-Versions-hc-35 (247 bytes)
>   INFO 13:59:07,815 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-32 (490 bytes)
>   INFO 13:59:07,816 Opening /opt/cassandra/database/data/system/IndexInfo/system-IndexInfo-hc-33 (115 bytes)
>   INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-9 (506 bytes)
>   INFO 13:59:07,850 Opening /opt/cassandra/database/data/system/schema_keyspaces/system-schema_keyspaces-hc-10 (506 bytes)
>   INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-74 (148 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-72 (406 bytes) INFO 13:59:07,867 Opening /opt/cassandra/database/data/system/LocationInfo/system-LocationInfo-hc-73 (80 bytes)ERROR 13:59:08,244 Exception encountered during startup
> java.lang.IllegalArgumentException: value already present: 1034
>          at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
>          at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
>          at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
>          at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
>          at org.apache.cassandra.config.Schema.load(Schema.java:385)
>          at org.apache.cassandra.config.Schema.load(Schema.java:106)
>          at org.apache.cassandra.config.Schema.load(Schema.java:91)
>          at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
>          at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> java.lang.IllegalArgumentException: value already present: 1034
>          at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
>          at com.google.common.collect.AbstractBiMap.putInBothMaps(AbstractBiMap.java:111)
>          at com.google.common.collect.AbstractBiMap.put(AbstractBiMap.java:96)
>          at com.google.common.collect.HashBiMap.put(HashBiMap.java:84)
>          at org.apache.cassandra.config.Schema.load(Schema.java:385)
>          at org.apache.cassandra.config.Schema.load(Schema.java:106)
>          at org.apache.cassandra.config.Schema.load(Schema.java:91)
>          at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
>          at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353)
>          at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106)
> Exception encountered during startup: value already present: 1034
>
>
> What can I do to get my cluster back online?
>
> Thank you
>
> Christoph Eberhardt
>
>
>
>