You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Arend-Jan Wijtzes <aj...@wise-guys.nl> on 2012/08/08 15:03:52 UTC
lost keyspace
Hi,
Today we rebooted a node in our cluster for maintenance and after that
one of the keyspaces went missing. This is what we did leading up to
this:
nodetool drain -h localhost
reboot
restart cassandra
There was an error during startup:
ERROR [MutationStage:109] 2012-08-08 14:05:56,443 RowMutationVerbHandler.java (line 61) Error in row mutation org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find cfId=1014
(then we found keyspace twitter is missing)
The sstables are still there so I tried to re-create the keyspace:
cassandra-cli -h localhost
[default@unknown] create keyspace twitter with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:3} ;
b890b65a-40a6-3fc5-838d-34d58368cb64
Waiting for schema agreement...
... schemas agree across the cluster
NotFoundException()
default@unknown] use twitter;
Keyspace 'twitter' not found.
So we can't create the keyspace. The cassandra version is 1.1.0.
What is the proper way to deal with this?
Thanks for your help.
Arend-Jan
--
Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl
Re: lost keyspace
Posted by Arend-Jan Wijtzes <aj...@wise-guys.nl>.
On Wed, Aug 08, 2012 at 05:08:56PM +0200, Mateusz Korniak wrote:
> On Wednesday 08 of August 2012, Arend-Jan Wijtzes wrote:
> > Forgot to mention that the keyspace 'twitter' was created, then droppend
> > and re-created a couple of days ago.
> >
> > How about if I create a new keyspace with the same definition and then
> > copy the existing tables into the proper place and call nodetool
> > refresh on each node. Would that work or are there references in the
> > tables to the keyspace name?
>
> Worth to try, should work. Perhaps use _copy_ of SStables.
> I did that during schema version desync solving few times on 1.0.x.
> Make sure you have same schema version on all nodes first.
OK that worked. One extra step was required and that was to rename the
keyspace part in the filenames of the sstables.
Still would like to know why the original keyspace got lost and if there
is a 'proper' way to restore it.
--
Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl
Re: lost keyspace
Posted by Mateusz Korniak <ma...@ant.gliwice.pl>.
On Wednesday 08 of August 2012, Arend-Jan Wijtzes wrote:
> Forgot to mention that the keyspace 'twitter' was created, then droppend
> and re-created a couple of days ago.
>
> How about if I create a new keyspace with the same definition and then
> copy the existing tables into the proper place and call nodetool
> refresh on each node. Would that work or are there references in the
> tables to the keyspace name?
Worth to try, should work. Perhaps use _copy_ of SStables.
I did that during schema version desync solving few times on 1.0.x.
Make sure you have same schema version on all nodes first.
Regards,
--
Mateusz Korniak
Re: lost keyspace
Posted by Arend-Jan Wijtzes <aj...@wise-guys.nl>.
Forgot to mention that the keyspace 'twitter' was created, then droppend
and re-created a couple of days ago.
How about if I create a new keyspace with the same definition and then
copy the existing tables into the proper place and call nodetool
refresh on each node. Would that work or are there references in the
tables to the keyspace name?
On Wed, Aug 08, 2012 at 03:03:52PM +0200, Arend-Jan Wijtzes wrote:
> Hi,
>
> Today we rebooted a node in our cluster for maintenance and after that
> one of the keyspaces went missing. This is what we did leading up to
> this:
>
> nodetool drain -h localhost
> reboot
>
> restart cassandra
> There was an error during startup:
> ERROR [MutationStage:109] 2012-08-08 14:05:56,443 RowMutationVerbHandler.java (line 61) Error in row mutation org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find cfId=1014
>
> (then we found keyspace twitter is missing)
>
> The sstables are still there so I tried to re-create the keyspace:
> cassandra-cli -h localhost
> [default@unknown] create keyspace twitter with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:3} ;
> b890b65a-40a6-3fc5-838d-34d58368cb64
> Waiting for schema agreement...
> ... schemas agree across the cluster
> NotFoundException()
>
> default@unknown] use twitter;
> Keyspace 'twitter' not found.
>
> So we can't create the keyspace. The cassandra version is 1.1.0.
> What is the proper way to deal with this?
>
> Thanks for your help.
>
> Arend-Jan
> --
> Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl
--
Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl