You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@cassandra.apache.org by "Thibaut (JIRA)" <ji...@apache.org> on 2011/01/31 10:16:11 UTC

[jira] Created: (CASSANDRA-2076) Not starting due to Invalid saved cache

Not starting due to Invalid saved cache
---------------------------------------

                 Key: CASSANDRA-2076
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
             Project: Cassandra
          Issue Type: Bug
          Components: Core
         Environment: linux
            Reporter: Thibaut
            Priority: Minor
             Fix For: 0.7.2


This occured on two nodes on me (running 0.7.1 from svn)

One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)

The saved_cache was corrupt afterwards and I couldn't start the nodes. 

After deleting the saved_caches directory I could start the nodes again. 

Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?




 INFO 22:31:11,570 reading saved cache
/hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
ERROR 22:31:11,595 Exception encountered during startup.
java.lang.RuntimeException: The provided key was not UTF8 encoded.
       at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
       at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
       at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
       at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
       at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
       at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
       at org.apache.cassandra.db.Table.initCf(Table.java:360)
       at org.apache.cassandra.db.Table.<init>(Table.java:290)
       at org.apache.cassandra.db.Table.open(Table.java:107)
       at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
       at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
       at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
       at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
       at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
       at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
       at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
       ... 11 more
Exception encountered during startup.
java.lang.RuntimeException: The provided key was not UTF8 encoded.
       at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
       at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
       at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
       at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
       at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
       at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
       at org.apache.cassandra.db.Table.initCf(Table.java:360)
       at org.apache.cassandra.db.Table.<init>(Table.java:290)
       at org.apache.cassandra.db.Table.open(Table.java:107)
       at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
       at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
       at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
       at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
       at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
       at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
       at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
       ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not starting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988780#comment-12988780 ] 

Thibaut commented on CASSANDRA-2076:
------------------------------------

This might be related:

Two other nodes (still running) also show up the "The provided key was not UTF8 encoded." error in the log.

I have never seen this error in 0.7.0


ERROR [MutationStage:19] 2011-01-30 21:36:16,951 DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor
java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
        at org.apache.cassandra.db.Table.apply(Table.java:406)
        at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:190)
        at org.apache.cassandra.service.StorageProxy$2.runMayThrow(StorageProxy.java:288)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 3 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
        ... 8 more
ERROR [MutationStage:19] 2011-01-30 21:36:16,991 AbstractCassandraDaemon.java (line 119) Fatal exception in thread Thread[MutationStage:19,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
        at org.apache.cassandra.db.Table.apply(Table.java:406)
        at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:190)
        at org.apache.cassandra.service.StorageProxy$2.runMayThrow(StorageProxy.java:288)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 3 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
        ... 8 more
 WARN [ScheduledTasks:1] 2011-01-30 21:36:21,450 MessagingService.java (line 506) Dropped 8 MUTATION messages in the last 5000ms





> Not starting due to Invalid saved cache
> ---------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Minor
>             Fix For: 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew F. Dennis reassigned CASSANDRA-2076:
--------------------------------------------

    Assignee: Matthew F. Dennis

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990491#comment-12990491 ] 

Thibaut commented on CASSANDRA-2076:
------------------------------------

I was running yesterday's version with the Consitency fix and getting a similar error messages while reading the commitlog at start. I had to delete the commitlog (data loss) to restart cassandra.

Is this also related to CASSANDRA-2102 or shall I open a new bug report?


 INFO 23:50:00,922 Replaying /hd1/cassandra_md5/commitlog/CommitLog-1296731219671.log, /hd1/cassandra_md5/commitlog/CommitLog-1296759540149.log, /hd1/cassandra_md5/commitlog/CommitLog-1296760444366.log, /hd1/cassandra_md5/commitlog/CommitLog-1296761120546.log, /hd1/cassandra_md5/commitlog/CommitLog-1296762054192.log, /hd1/cassandra_md5/commitlog/CommitLog-1296773137101.log, /hd1/cassandra_md5/commitlog/CommitLog-1296773242671.log
 INFO 23:50:02,192 Finished reading /hd1/cassandra_md5/commitlog/CommitLog-1296731219671.log
ERROR 23:50:02,235 Fatal exception in thread Thread[MutationStage:7,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
        at org.apache.cassandra.db.Table.apply(Table.java:406)
        at org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:294)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 6 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
        ... 10 more
 INFO 23:50:02,245 Finished reading /hd1/cassandra_md5/commitlog/CommitLog-1296759540149.log
ERROR 23:50:02,245 Exception encountered during startup.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:455)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:159)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:166)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:307)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:451)
        ... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
        at org.apache.cassandra.db.Table.apply(Table.java:406)
        at org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:294)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 6 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
        ... 10 more
Exception encountered during startup.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:455)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:301)
        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:159)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:166)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:307)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:451)
        ... 5 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: The provided key was not UTF8 encoded.
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
        at org.apache.cassandra.db.Table.apply(Table.java:406)
        at org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:294)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 6 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
        ... 10 more




> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-2076:
--------------------------------------

    Remaining Estimate: 2h
     Original Estimate: 2h

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew F. Dennis reassigned CASSANDRA-2076:
--------------------------------------------

    Assignee: Matthew F. Dennis  (was: Jonathan Ellis)

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew F. Dennis updated CASSANDRA-2076:
-----------------------------------------

    Attachment: 2076-cassandra-0.7.txt

attached patch (depends on CASSANDRA-2102) allows C* to start when the saved caches are invalid/corrupt.

CASSANDRA-2102 prevents the saved caches from becoming corrupt in the first place.



> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Issue Comment Edited: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990643#comment-12990643 ] 

Thibaut edited comment on CASSANDRA-2076 at 2/4/11 5:41 PM:
------------------------------------------------------------

I can't say for sure. I guess so.

Can you trigger a hudson build with the most recent fixes as well. (Don't know for sure which revision to take) So I can let it run over the weekend and check if any of the exceptions and bugs (these ones, cpu spike, consistency...) occurs again or not.




      was (Author: tbritz):
    I can't say for sure. I guess so.

Can you trigger a hudson build with the most recent fixes as well. (Don't know for sure which revision to take) So I can let it run over the weekend and check if the exception occurs again or not.



  
> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990583#comment-12990583 ] 

Jonathan Ellis commented on CASSANDRA-2076:
-------------------------------------------

I reverted CASSANDRA-1743 which is what caused CASSANDRA-2102.  I suspect it's causing this other problem too for reasons not yet understood.

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990171#comment-12990171 ] 

Jonathan Ellis commented on CASSANDRA-2076:
-------------------------------------------

The problem is that we're not cloning the buffer backing the key read from thrift, so we are hitting a problem similar to CASSANDRA-1743.  Matt is working on a fix.

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not starting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thibaut updated CASSANDRA-2076:
-------------------------------

    Fix Version/s: 0.7.1

> Not starting due to Invalid saved cache
> ---------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew F. Dennis updated CASSANDRA-2076:
-----------------------------------------

    Attachment:     (was: 2076-cassandra-0.7.txt)

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-2076:
--------------------------------------

    Attachment: 2076-v2.txt

v2 is a less-invasive fix to the original problem.

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew F. Dennis updated CASSANDRA-2076:
-----------------------------------------

    Attachment: 2076-cassandra-0.7.txt

I like the first patch better (recently rebased) as it adds a version field at the top and operates on an actual count instead of using in.available() which is not actually reliable in all cases.

In addition the rebase fixes a recently introduced bug that prevented the caches from getting saved where BufferedRandomAccessFile doesn't accept "w" as a mode and adds some comments in CacheWriter about how the totalBytes reported via ICompactionInfo is just an approximation.

If we don't want these other changes, +1 on the v2 patch.

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990580#comment-12990580 ] 

Jonathan Ellis commented on CASSANDRA-2076:
-------------------------------------------

It's a different bug.

Did you have any nodes down for part of the time (any hinted handoff going on?)

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990695#comment-12990695 ] 

Jonathan Ellis commented on CASSANDRA-2076:
-------------------------------------------

build scheduled

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thibaut updated CASSANDRA-2076:
-------------------------------

    Summary: Not restarting due to Invalid saved cache  (was: Not starting due to Invalid saved cache)

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989160#comment-12989160 ] 

Thibaut commented on CASSANDRA-2076:
------------------------------------

This just happens if you kill the node with -9

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not starting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988781#comment-12988781 ] 

Thibaut commented on CASSANDRA-2076:
------------------------------------

I brought down and restarted the entire cluster. (100 nodes, 5x20 nodes)

Every single node complains of an invalid file in the saved_cache directory.

> Not starting due to Invalid saved cache
> ---------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis updated CASSANDRA-2076:
--------------------------------------

    Reviewer: mdennis

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Hudson (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992861#comment-12992861 ] 

Hudson commented on CASSANDRA-2076:
-----------------------------------

Integrated in Cassandra-0.7 #273 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/273/])
    continue starting when invalid savedcache entries are encountered
patch by mdennis and jbellis for CASSANDRA-2076


> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Assigned: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis reassigned CASSANDRA-2076:
-----------------------------------------

    Assignee: Jonathan Ellis  (was: Matthew F. Dennis)

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt, 2076-v2.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Updated: (CASSANDRA-2076) Not starting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thibaut updated CASSANDRA-2076:
-------------------------------

    Priority: Critical  (was: Minor)

> Not starting due to Invalid saved cache
> ---------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: linux
>            Reporter: Thibaut
>            Priority: Critical
>             Fix For: 0.7.2
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990895#comment-12990895 ] 

Matthew F. Dennis commented on CASSANDRA-2076:
----------------------------------------------

the commit logs have checksums in them to attempt to catch corrupted files.  C* is supposed to log that it is corrupted, skip over it and then continue.  The fact that *any* value inside a commit log can cause C* not to start is a bug of it's own.  I've opened CASSANDRA-2113 to track this.

> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Commented: (CASSANDRA-2076) Not restarting due to Invalid saved cache

Posted by "Thibaut (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/CASSANDRA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990643#comment-12990643 ] 

Thibaut commented on CASSANDRA-2076:
------------------------------------

I can't say for sure. I guess so.

Can you trigger a hudson build with the most recent fixes as well. (Don't know for sure which revision to take) So I can let it run over the weekend and check if the exception occurs again or not.




> Not restarting due to Invalid saved cache
> -----------------------------------------
>
>                 Key: CASSANDRA-2076
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2076
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>         Environment: linux
>            Reporter: Thibaut
>            Assignee: Matthew F. Dennis
>            Priority: Critical
>             Fix For: 0.7.1, 0.7.2
>
>         Attachments: 2076-cassandra-0.7.txt
>
>
> This occured on two nodes on me (running 0.7.1 from svn)
> One node was killed by the kernel due to a OOM and the other node was haning and I had to kill it manually with kill -9 (kill didn't work). (maybe these were faulty hardware nodes, I don't know)
> The saved_cache was corrupt afterwards and I couldn't start the nodes. 
> After deleting the saved_caches directory I could start the nodes again. 
> Instead of not starting when an error occurs, cassandra could simply delete the errornous file and continue to start?
>  INFO 22:31:11,570 reading saved cache
> /hd1/cassandra_md5/saved_caches/table_attributes-table_attributes-KeyCache
> ERROR 22:31:11,595 Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more
> Exception encountered during startup.
> java.lang.RuntimeException: The provided key was not UTF8 encoded.
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:159)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.decorateKey(OrderPreservingPartitioner.java:44)
>        at org.apache.cassandra.db.ColumnFamilyStore.readSavedCache(ColumnFamilyStore.java:281)
>        at org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:218)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:458)
>        at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:440)
>        at org.apache.cassandra.db.Table.initCf(Table.java:360)
>        at org.apache.cassandra.db.Table.<init>(Table.java:290)
>        at org.apache.cassandra.db.Table.open(Table.java:107)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:167)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:312)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:81)
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
>        at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:781)
>        at org.apache.cassandra.utils.FBUtilities.decodeToUTF8(FBUtilities.java:403)
>        at org.apache.cassandra.dht.OrderPreservingPartitioner.getToken(OrderPreservingPartitioner.java:155)
>        ... 11 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira