You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Sandeep Varupula (Jira)" <ji...@apache.org> on 2020/06/19 04:08:00 UTC

[jira] [Commented] (CASSANDRA-15804) system_schema keyspace complain of schema mismatch during upgrade

    [ https://issues.apache.org/jira/browse/CASSANDRA-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140191#comment-17140191 ] 

Sandeep Varupula commented on CASSANDRA-15804:
----------------------------------------------

I faced same issue while upgrading from 3.11.3 to 3.11.6. I agree with above description. We need to handle error better. 
{code:java}
// DEBUG [NonPeriodicTasks:1] 2020-06-18 13:55:13,490 MigrationManager.java:142 - submitting migration task for /xx.xx.xx.xx, schema version mismatch: local/real=efc0553c-f5af-3a7c-ae2b-59403c
3aa9a1, local/compatible=313561f1-3294-3395-ba8e-81b712ecf7e9, remote=e14e388e-f3e6-34ba-9067-8379f6c4f012
DEBUG [NonPeriodicTasks:1] 2020-06-18 13:55:13,491 MigrationManager.java:142 - submitting migration task for /xx.xx.xx.xx, schema version mismatch: local/real=efc0553c-f5af-3a7c-ae2b-59403c
3aa9a1, local/compatible=313561f1-3294-3395-ba8e-81b712ecf7e9, remote=e14e388e-f3e6-34ba-9067-8379f6c4f012


ERROR [MessagingService-Incoming-/xx.xx.xx.xx] 2020-06-18 13:55:13,502 CassandraDaemon.java:228 - Exception in thread Thread[MessagingService-Incoming-/xx.xx.xx.xx,5,main]
java.lang.RuntimeException: Unknown column kind during deserialization
        at org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:452) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:412) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:669) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:652) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180) ~[apache-cassandra-3.11.3.jar:3.11.3]
        at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94) ~[apache-cassandra-3.11.3.jar:3.11.3]
 {code}

> system_schema keyspace complain of schema mismatch during upgrade
> -----------------------------------------------------------------
>
>                 Key: CASSANDRA-15804
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15804
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Pedro Gordo
>            Priority: Low
>
> When upgrading from 3.11.4 to 3.11.6, we got the following error:
> {code:Plain Text}
> ERROR [MessagingService-Incoming-/10.20.11.59] 2020-05-07 13:53:52,627 CassandraDaemon.java:228 - Exception in thread Thread[MessagingService-Incoming-/10.20.11.59,5,main]
> java.lang.RuntimeException: Unknown column kind during deserialization
>     at org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:464) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.SerializationHeader$Serializer.deserializeForMessaging(SerializationHeader.java:419) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.deserializeHeader(UnfilteredRowIteratorSerializer.java:195) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:851) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:839) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:425) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:434) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:675) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.service.MigrationManager$MigrationsSerializer.deserialize(MigrationManager.java:658) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:192) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:180) ~[apache-cassandra-3.11.4.jar:3.11.4]
>     at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:94) ~[apache-cassandra-3.11.4.jar:3.11.4]
> {code}
> I've noticed that system_schema.dropped_columns has a new column called "kind".
> No issues arise from this error message, and the error disappeared after upgrading all nodes. But it still caused concerns due to the ERROR logging level, although "nodetool describecluster" reported only one schema version.
> It makes sense for the system keyspaces to not be included for the "describecluster" schema version check, but it seems to me that these internal schema mismatches should be ignored if the versions are different between the nodes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org