You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David Hawthorne (JIRA)" <ji...@apache.org> on 2012/05/10 01:19:52 UTC

[jira] [Created] (CASSANDRA-4235) keyspaces missing after restart

David Hawthorne created CASSANDRA-4235:
------------------------------------------

             Summary: keyspaces missing after restart
                 Key: CASSANDRA-4235
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
             Project: Cassandra
          Issue Type: Bug
          Components: Core
    Affects Versions: 1.1.0
         Environment: ubuntu
            Reporter: David Hawthorne
            Priority: Critical


After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.

The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273628#comment-13273628 ] 

Pavel Yaskevich commented on CASSANDRA-4235:
--------------------------------------------

Yeah, that is fixed by CASSANDRA-4219 too
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273259#comment-13273259 ] 

Steeve Morin commented on CASSANDRA-4235:
-----------------------------------------

I should add, don't mind asking questions, I'll be keeping Cassandra in this state if you guys need to inquire.
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Resolved] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pavel Yaskevich resolved CASSANDRA-4235.
----------------------------------------

    Resolution: Duplicate

This is the same problem as CASSANDRA-4219
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Issue Comment Edited] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273588#comment-13273588 ] 

Steeve Morin edited comment on CASSANDRA-4235 at 5/11/12 8:35 PM:
------------------------------------------------------------------

Would love but won't be able for at least a few days...
EDIT: Also, please note that the keyspace doesn't load to begin with, even tho the files are present.
                
      was (Author: steeve):
    Would love but won't be able for at least a few days...
                  
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273497#comment-13273497 ] 

Pavel Yaskevich commented on CASSANDRA-4235:
--------------------------------------------

Steeve, can you please apply a patch from CASSANDRA-4219 and check if it fixes your situation?
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Comment Edited] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13275791#comment-13275791 ] 

Pavel Yaskevich edited comment on CASSANDRA-4235 at 5/15/12 2:34 PM:
---------------------------------------------------------------------

Steeve, as CASSANDRA-4219 is now resolved, can you please test if that fixes your problem too? If so we can close this one as duplicate.
                
      was (Author: xedin):
    Steeve, as CASSANDRA-4219 is not resolved, can you please test if that fixes your problem too? If so we can close this one as duplicate.
                  
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273254#comment-13273254 ] 

Steeve Morin commented on CASSANDRA-4235:
-----------------------------------------

Happened to me a few times. I thought I did something wrong. Had 3 keyspaces (not counting system), only 1 is showing up now. The files are still here.

To complement the previous file, here a stack trace when trying to delete/re-create the missing keyspaces:
{{
 INFO 15:33:43,505 Enqueuing flush of Memtable-schema_keyspaces@1108352425(192/240 serialized/live bytes, 4 ops)
 INFO 15:33:43,505 Writing Memtable-schema_keyspaces@1108352425(192/240 serialized/live bytes, 4 ops)
 INFO 15:33:43,512 Completed flushing /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-36-Data.db (256 bytes)
 INFO 15:33:43,513 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-34-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-35-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-36-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-33-Data.db')]
 INFO 15:33:43,524 Compacted to [/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-37-Data.db,].  1,498 to 730 (~48% of original) bytes for 3 keys at 0.069618MB/s.  Time: 10ms.
ERROR 15:33:43,525 Exception in thread Thread[MigrationStage:1,5,main]
java.lang.AssertionError
	at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441)
	at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339)
	at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269)
	at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:680)
ERROR 15:33:43,525 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
	at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
	at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
	at org.apache.cassandra.service.MigrationManager.announceNewKeyspace(MigrationManager.java:129)
	at org.apache.cassandra.thrift.CassandraServer.system_add_keyspace(CassandraServer.java:987)
	at org.apache.cassandra.thrift.Cassandra$Processor$system_add_keyspace.getResult(Cassandra.java:3370)
	at org.apache.cassandra.thrift.Cassandra$Processor$system_add_keyspace.getResult(Cassandra.java:3358)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
	at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
	at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
	... 11 more
Caused by: java.lang.AssertionError
	at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441)
	at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339)
	at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269)
	at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	... 3 more
}}
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Issue Comment Edited] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273254#comment-13273254 ] 

Steeve Morin edited comment on CASSANDRA-4235 at 5/11/12 1:38 PM:
------------------------------------------------------------------

Happened to me a few times. I thought I did something wrong. Had 3 keyspaces (not counting system), only 1 is showing up now. The files are still here.

To complement the previous file, here a stack trace when trying to delete/re-create the missing keyspaces: https://gist.github.com/2659696
                
      was (Author: steeve):
    Happened to me a few times. I thought I did something wrong. Had 3 keyspaces (not counting system), only 1 is showing up now. The files are still here.

To complement the previous file, here a stack trace when trying to delete/re-create the missing keyspaces:
{{
 INFO 15:33:43,505 Enqueuing flush of Memtable-schema_keyspaces@1108352425(192/240 serialized/live bytes, 4 ops)
 INFO 15:33:43,505 Writing Memtable-schema_keyspaces@1108352425(192/240 serialized/live bytes, 4 ops)
 INFO 15:33:43,512 Completed flushing /var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-36-Data.db (256 bytes)
 INFO 15:33:43,513 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-34-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-35-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-36-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-33-Data.db')]
 INFO 15:33:43,524 Compacted to [/var/lib/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-hc-37-Data.db,].  1,498 to 730 (~48% of original) bytes for 3 keys at 0.069618MB/s.  Time: 10ms.
ERROR 15:33:43,525 Exception in thread Thread[MigrationStage:1,5,main]
java.lang.AssertionError
	at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441)
	at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339)
	at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269)
	at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:680)
ERROR 15:33:43,525 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError
	at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
	at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
	at org.apache.cassandra.service.MigrationManager.announceNewKeyspace(MigrationManager.java:129)
	at org.apache.cassandra.thrift.CassandraServer.system_add_keyspace(CassandraServer.java:987)
	at org.apache.cassandra.thrift.Cassandra$Processor$system_add_keyspace.getResult(Cassandra.java:3370)
	at org.apache.cassandra.thrift.Cassandra$Processor$system_add_keyspace.getResult(Cassandra.java:3358)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
	at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
	at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
	... 11 more
Caused by: java.lang.AssertionError
	at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441)
	at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339)
	at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269)
	at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	... 3 more
}}
                  
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis reassigned CASSANDRA-4235:
-----------------------------------------

    Assignee: Pavel Yaskevich
    
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273588#comment-13273588 ] 

Steeve Morin commented on CASSANDRA-4235:
-----------------------------------------

Would love but won't be able for at least a few days...
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273662#comment-13273662 ] 

Steeve Morin commented on CASSANDRA-4235:
-----------------------------------------

Wow, that sure was a quick patch! Any plans on when 1.1.1 will be released?
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Steeve Morin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steeve Morin updated CASSANDRA-4235:
------------------------------------

    Attachment: cassandra2.log

Complete start log and stack trace, with DEBUG.
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13273267#comment-13273267 ] 

Brandon Williams commented on CASSANDRA-4235:
---------------------------------------------

Looks like this is related to CASSANDRA-4219
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pantelis Sopasakis (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281449#comment-13281449 ] 

Pantelis Sopasakis commented on CASSANDRA-4235:
-----------------------------------------------

Thanks to the people in #cassandra on IRC I managed to find out how to recover from such an incident. Here's the procedure: 

0. Before it is too late export your KS schema: Create a file called 'schema_export' and add the following lines inside:
 use MyKeyspace;
 show schema;
then run this cassandra script and save the output in another file:
 cassandra-cli < schema_export > my.schema
keep this file under lock and key! If you change your schema, remember to update the file. Edit the file and remove non-command lines like "Connected to: "Test Cluster" on 127.0.0.1/9160".

1. Stop the server
1. Delete 'system': rm -r /var/lib/cassandra/data/system
2. Start the server
3. Load the schema: cassandra-cli < my.schema
4. Restart the server and connect using the client. Apply a 'use MyKeyspace' and you'll see that your keyspace is there. Also your data is there.

I hope this helps.
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (CASSANDRA-4235) keyspaces missing after restart

Posted by "Pavel Yaskevich (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13275791#comment-13275791 ] 

Pavel Yaskevich commented on CASSANDRA-4235:
--------------------------------------------

Steeve, as CASSANDRA-4219 is not resolved, can you please test if that fixes your problem too? If so we can close this one as duplicate.
                
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Assignee: Pavel Yaskevich
>            Priority: Critical
>         Attachments: cassandra.log, cassandra2.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (CASSANDRA-4235) keyspaces missing after restart

Posted by "David Hawthorne (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CASSANDRA-4235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Hawthorne updated CASSANDRA-4235:
---------------------------------------

    Attachment: cassandra.log
    
> keyspaces missing after restart
> -------------------------------
>
>                 Key: CASSANDRA-4235
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4235
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.0
>         Environment: ubuntu
>            Reporter: David Hawthorne
>            Priority: Critical
>         Attachments: cassandra.log
>
>
> After restarting cassandra, the only keyspace that exists is the system keyspace.  All data files are still there for other keyspaces, though.
> The issue was also described here:  http://www.mail-archive.com/user@cassandra.apache.org/msg22231.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira