You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Hiller, Dean" <De...@nrel.gov> on 2013/06/24 14:42:17 UTC
AssertionError: Unknown keyspace?
I haven't seen this error in a long time. We just received the below error in production when rebuilding a node…any ideas on how to get around this? We had rebuilt 3 other nodes already I think(we have been swapping hardware)
ERROR 06:32:21,474 Exception in thread Thread[ReadStage:1,5,main]
java.lang.AssertionError: Unknown keyspace databus5
at org.apache.cassandra.db.Table.<init>(Table.java:263)
at org.apache.cassandra.db.Table.open(Table.java:110)
at org.apache.cassandra.db.Table.open(Table.java:88)
at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Thanks for any insight,
Dean
Re: AssertionError: Unknown keyspace?
Posted by "Hiller, Dean" <De...@nrel.gov>.
Yes, it would be nice at startup just to say don't list your seed node as this node and then fail out and we would have known this a long long time ago ;).
Dean
From: Wei Zhu <wz...@yahoo.com>>
Reply-To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>, Wei Zhu <wz...@yahoo.com>>
Date: Monday, June 24, 2013 12:36 PM
To: "user@cassandra.apache.org<ma...@cassandra.apache.org>" <us...@cassandra.apache.org>>
Subject: Re: AssertionError: Unknown keyspace?
I have got bitten by it once. At least there should be a message saying, there is no streaming data since it's a seed node.
I searched the source code, the message was there and it got removed at certain version.
-Wei
________________________________
From: Robert Coli <rc...@eventbrite.com>>
To: user@cassandra.apache.org<ma...@cassandra.apache.org>
Sent: Monday, June 24, 2013 10:34 AM
Subject: Re: AssertionError: Unknown keyspace?
On Mon, Jun 24, 2013 at 6:04 AM, Hiller, Dean <De...@nrel.gov>> wrote:
> Oh shoot, this is a seed node. Is there documentation on how to bootstrap
> a seed node? If I have seeds of A, B, C for every machine on the ring and
> I am bootstrapping node B, do I just modify cassandra.yaml and remove node
> B from the yaml file temporarily and boot it up
Yes. The only thing that makes a node fail that check is being in its
own seed list. But if the node is in other nodes' seed lists, those
nodes will contact it anyway. This strongly implies that the
"contains()" check there is the wrong test, but I've never nailed that
down and/or filed a ticket on it. Conversation at the summit suggests
I should, making a note to do so...
=Rob
Re: AssertionError: Unknown keyspace?
Posted by Wei Zhu <wz...@yahoo.com>.
I have got bitten by it once. At least there should be a message saying, there is no streaming data since it's a seed node.
I searched the source code, the message was there and it got removed at certain version.
-Wei
________________________________
From: Robert Coli <rc...@eventbrite.com>
To: user@cassandra.apache.org
Sent: Monday, June 24, 2013 10:34 AM
Subject: Re: AssertionError: Unknown keyspace?
On Mon, Jun 24, 2013 at 6:04 AM, Hiller, Dean <De...@nrel.gov> wrote:
> Oh shoot, this is a seed node. Is there documentation on how to bootstrap
> a seed node? If I have seeds of A, B, C for every machine on the ring and
> I am bootstrapping node B, do I just modify cassandra.yaml and remove node
> B from the yaml file temporarily and boot it up
Yes. The only thing that makes a node fail that check is being in its
own seed list. But if the node is in other nodes' seed lists, those
nodes will contact it anyway. This strongly implies that the
"contains()" check there is the wrong test, but I've never nailed that
down and/or filed a ticket on it. Conversation at the summit suggests
I should, making a note to do so...
=Rob
Re: AssertionError: Unknown keyspace?
Posted by Robert Coli <rc...@eventbrite.com>.
On Mon, Jun 24, 2013 at 6:04 AM, Hiller, Dean <De...@nrel.gov> wrote:
> Oh shoot, this is a seed node. Is there documentation on how to bootstrap
> a seed node? If I have seeds of A, B, C for every machine on the ring and
> I am bootstrapping node B, do I just modify cassandra.yaml and remove node
> B from the yaml file temporarily and boot it up
Yes. The only thing that makes a node fail that check is being in its
own seed list. But if the node is in other nodes' seed lists, those
nodes will contact it anyway. This strongly implies that the
"contains()" check there is the wrong test, but I've never nailed that
down and/or filed a ticket on it. Conversation at the summit suggests
I should, making a note to do so...
=Rob
Re: AssertionError: Unknown keyspace?
Posted by "Hiller, Dean" <De...@nrel.gov>.
Oh shoot, this is a seed node. Is there documentation on how to bootstrap
a seed node? If I have seeds of A, B, C for every machine on the ring and
I am bootstrapping node B, do I just modify cassandra.yaml and remove node
B from the yaml file temporarily and boot it up (Note, I still received
the unknown keyspace errors :( but it is bootstrapping now) and I assume I
can add node B back once all the data is in there.
Thanks,
Dean
On 6/24/13 6:55 AM, "Hiller, Dean" <De...@nrel.gov> wrote:
>Ah, so digging deeper, it is not bootstrapping. How do I force the node
>to bootstrap? (this is version 1.2.2 and the other nodes somehow knew to
>bootstrap automatically but this one I need to force for some reason). I
>remember there was a property for this.
>
>NOTE: I enabled some debug logs and auto bootstrap is true according to
>this log
>DEBUG 06:53:03,411 setting auto_bootstrap to true
>
>OR better yet, if someone can point me to the code on where bootstrap is
>decided so I can see why it decides not to bootstrap?
>
>Thanks,
>Dean
>
>On 6/24/13 6:42 AM, "Hiller, Dean" <De...@nrel.gov> wrote:
>
>>I haven't seen this error in a long time. We just received the below
>>error in production when rebuilding a nodeŠany ideas on how to get around
>>this? We had rebuilt 3 other nodes already I think(we have been swapping
>>hardware)
>>
>>ERROR 06:32:21,474 Exception in thread Thread[ReadStage:1,5,main]
>>java.lang.AssertionError: Unknown keyspace databus5
>>at org.apache.cassandra.db.Table.<init>(Table.java:263)
>>at org.apache.cassandra.db.Table.open(Table.java:110)
>>at org.apache.cassandra.db.Table.open(Table.java:88)
>>at
>>org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
>>at
>>org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java
>>:
>>56)
>>at
>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor
>>.
>>java:895)
>>at
>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav
>>a
>>:918)
>>at java.lang.Thread.run(Thread.java:662)
>>
>>Thanks for any insight,
>>Dean
>
Re: AssertionError: Unknown keyspace?
Posted by "Hiller, Dean" <De...@nrel.gov>.
Ah, so digging deeper, it is not bootstrapping. How do I force the node
to bootstrap? (this is version 1.2.2 and the other nodes somehow knew to
bootstrap automatically but this one I need to force for some reason). I
remember there was a property for this.
NOTE: I enabled some debug logs and auto bootstrap is true according to
this log
DEBUG 06:53:03,411 setting auto_bootstrap to true
OR better yet, if someone can point me to the code on where bootstrap is
decided so I can see why it decides not to bootstrap?
Thanks,
Dean
On 6/24/13 6:42 AM, "Hiller, Dean" <De...@nrel.gov> wrote:
>I haven't seen this error in a long time. We just received the below
>error in production when rebuilding a nodeŠany ideas on how to get around
>this? We had rebuilt 3 other nodes already I think(we have been swapping
>hardware)
>
>ERROR 06:32:21,474 Exception in thread Thread[ReadStage:1,5,main]
>java.lang.AssertionError: Unknown keyspace databus5
>at org.apache.cassandra.db.Table.<init>(Table.java:263)
>at org.apache.cassandra.db.Table.open(Table.java:110)
>at org.apache.cassandra.db.Table.open(Table.java:88)
>at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
>at
>org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:
>56)
>at
>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.
>java:895)
>at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:918)
>at java.lang.Thread.run(Thread.java:662)
>
>Thanks for any insight,
>Dean