[jira] [Created] (CASSANDRA-18038) Protect against TableId mutability when system tables are created

["Abe Ratnofsky (Jira)" <ji...@apache.org>]

Abe Ratnofsky created CASSANDRA-18038:
-----------------------------------------

             Summary: Protect against TableId mutability when system tables are created
                 Key: CASSANDRA-18038
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-18038
             Project: Cassandra
          Issue Type: Improvement
          Components: Cluster/Schema
            Reporter: Abe Ratnofsky


In some situations, it may be possible to have non-deterministic CFIDs for system tables. This can happen if system tables were manually created prior to an upgrade (2.1->3.0, for example).

If non-deterministic CFIDs exist for system tables in system_schema.tables, then the cluster can operate correctly, as long as all members use the same CFIDs. This is because CFIDs are serialized as part of ReadCommands, which need to agree between coordinator and replica.

If a non-deterministic ID exists in system_schema.tables, then the instance upgrades to a Cassandra version including CASSANDRA-17044, the upgraded instance will move from the non-deterministic CFIDs to deterministic, causing "Unknown CF" exceptions for queries that span mixed-mode coordinators and replicas. In the case of AuthKeyspace, this can prevent users from authenticating and break all coordinated traffic to the upgraded node.

When system tables are set up, such as StorageService.doAuthSetup, the in-memory deterministic ID from AuthKeyspace.metadata() is used: [https://github.com/apache/cassandra/blob/cbafc28c5aec422e3b953eaa2f03a31d554312da/src/java/org/apache/cassandra/service/StorageService.java#L1242]

Then SchemaTransformations.updateSystemKeyspace will use the deterministic CFID calculated in-memory: [https://github.com/apache/cassandra/blob/cbafc28c5aec422e3b953eaa2f03a31d554312da/src/java/org/apache/cassandra/schema/SchemaTransformations.java#L193]

 

For clusters running without any manual intervention, there is no problem. But this does point to an opportunity to add more protection to our schema evolution code paths, to prevent CFID mutability from happening in the future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org