Re: Cassandra 2.1.18 - NPE during startup

[Anthony Grasso <an...@gmail.com>]

Hi Thomas,

The process you suggested to get around the issue should work with the
system.keyspaces table.

Make sure to backup the original *system.keyspaces* table files on the node
that fails to start. Then, copy only the *system.keyspaces *table files
from a working node into the *system/schema_keyspaces-...* folder of the
node that fails to start.

This method will only work for certain system tables as some of the data
stored in the system tables will differ between nodes.

Regards,
Anthony

On Thu, 28 Mar 2019 at 05:54, Steinmaurer, Thomas <
thomas.steinmaurer@dynatrace.com> wrote:

> Hello,
>
>
>
> any ideas regarding below, cause it happened again on a different node.
>
>
>
> Thanks
>
> Thomas
>
>
>
> *From:* Steinmaurer, Thomas <th...@dynatrace.com>
> *Sent:* Dienstag, 05. Februar 2019 23:03
> *To:* user@cassandra.apache.org
> *Subject:* Cassandra 2.1.18 - NPE during startup
>
>
>
> Hello,
>
>
>
> at a particular customer location, we are seeing the following NPE during
> startup with Cassandra 2.1.18.
>
>
>
> INFO  [SSTableBatchOpen:2] 2019-02-03 13:32:56,131 SSTableReader.java:475
> - Opening
> /var/opt/data/cassandra/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-130
> (256 bytes)
>
> ERROR [main] 2019-02-03 13:32:56,552 CassandraDaemon.java:583 - Exception
> encountered during startup
>
> org.apache.cassandra.io.FSReadError: java.lang.NullPointerException
>
>                 at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:672)
> ~[apache-cassandra-2.1.18.jar:2.1.18]
>
>                 at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:310)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
>                 at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:566)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
>                 at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:655)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
> Caused by: java.lang.NullPointerException: null
>
>                 at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:664)
> ~[apache-cassandra-2.1.18.jar:2.1.18]
>
>                 ... 3 common frames omitted
>
>
>
> I found https://issues.apache.org/jira/browse/CASSANDRA-10501
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCASSANDRA-10501&data=02%7C01%7Cthomas.steinmaurer%40dynatrace.com%7C796c92ba8c4d4c94937208d68bb69f1d%7C70ebe3a35b30435d9d677716d74ca190%7C1%7C0%7C636850013806426682&sdata=5Fx4An63c5R%2Faa2%2BfFXFCg7%2FDejG1V57st7UNtq9hrA%3D&reserved=0>,
> but this should be fixed in 2.1.18.
>
>
>
> Is the above log stating that it is caused by a system keyspace related
> SSTable?
>
>
>
> This is a 3 node setup with 2 others running fine. If system table related
> and as LocalStrategy is used as replication strategy (to my knowledge),
> perhaps simply copying over data for the schema_keyspaces table from
> another node might fix it?
>
>
>
> Any help appreciated.
>
>
>
> Thanks.
>
> Thomas
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
>