You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Poziombka, Wade L" <wa...@intel.com> on 2012/06/06 05:10:41 UTC
my devious QA - how to recover with power lost situation
They
1) setup a two node cluster and loaded 500K rows or something
2) add a third node, run nodetool move
3) while moving they pull the plug on the node
Cassandra won't start with the exception below. Now, this is obviously a very exceptional situation but the question is posed:
how best to recover this?
Observation:-
After the machine was up, the cassandra service failed to start. Following exception was observed in the cassandra logs:-
java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input length = 1
at
org.apache.cassandra.cql3.ColumnIdentifier.<init>(ColumnIdentifier.java:50)
at
org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
at org.apache.cassandra.cql3.CFDefinition.<init>(CFDefinition.java:59)
at
org.apache.cassandra.config.CFMetaData.updateCfDef(CFMetaData.java:1278)
at org.apache.cassandra.config.CFMetaData.keyAlias(CFMetaData.java:221)
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1162)
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1190)
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:272)
at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158)
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
at
org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:254)
at com.intel.soae.cassandra.server.SOAEDaemon.init(SOAEDaemon.java:435)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:207)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(Unknown Source)
at java.nio.charset.CharsetDecoder.decode(Unknown Source)
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:163)
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:120)
at
org.apache.cassandra.cql3.ColumnIdentifier.<init>(ColumnIdentifier.java:46)
... 18 more
Re: my devious QA - how to recover with power lost situation
Posted by Sylvain Lebresne <sy...@datastax.com>.
Looks like https://issues.apache.org/jira/browse/CASSANDRA-4269. I
suppose you were using 1.1.0. In which case that's fixed by 1.1.1.
--
Sylvain
On Wed, Jun 6, 2012 at 5:10 AM, Poziombka, Wade L
<wa...@intel.com> wrote:
> They
>
> 1) setup a two node cluster and loaded 500K rows or something
>
> 2) add a third node, run nodetool move
>
> 3) while moving they pull the plug on the node
>
>
>
> Cassandra won’t start with the exception below. Now, this is obviously a
> very exceptional situation but the question is posed:
>
>
>
> how best to recover this?
>
>
>
> Observation:-
>
> After the machine was up, the cassandra service failed to start. Following
> exception was observed in the cassandra logs:-
>
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
>
> at
>
> org.apache.cassandra.cql3.ColumnIdentifier.<init>(ColumnIdentifier.java:50)
>
> at
>
> org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
>
> at
> org.apache.cassandra.cql3.CFDefinition.<init>(CFDefinition.java:59)
>
> at
>
> org.apache.cassandra.config.CFMetaData.updateCfDef(CFMetaData.java:1278)
>
> at
> org.apache.cassandra.config.CFMetaData.keyAlias(CFMetaData.java:221)
>
> at
>
> org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1162)
>
> at
>
> org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1190)
>
> at
>
> org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
>
> at
>
> org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:272)
>
> at
> org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158)
>
> at
>
> org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:533)
>
> at
>
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
>
> at
>
> org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:254)
>
> at
> com.intel.soae.cassandra.server.SOAEDaemon.init(SOAEDaemon.java:435)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>
> at java.lang.reflect.Method.invoke(Unknown Source)
>
> at
>
> org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:207)
>
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>
> at java.nio.charset.CoderResult.throwException(Unknown Source)
>
> at java.nio.charset.CharsetDecoder.decode(Unknown Source)
>
> at
>
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:163)
>
> at
>
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:120)
>
> at
>
> org.apache.cassandra.cql3.ColumnIdentifier.<init>(ColumnIdentifier.java:46)
>
> ... 18 more
>
>
>
>