[jira] [Updated] (CASSANDRA-21549) Masked columns cannot be deserialized from cluster metadata, preventing node startup

["Stefan Miklosovic (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-21549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stefan Miklosovic updated CASSANDRA-21549:
------------------------------------------
    Reviewers: Stefan Miklosovic, Stefan Miklosovic  (was: Stefan Miklosovic)
               Stefan Miklosovic, Stefan Miklosovic  (was: Stefan Miklosovic)
       Status: Review In Progress  (was: Patch Available)

> Masked columns cannot be deserialized from cluster metadata, preventing node startup
> ------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-21549
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-21549
>             Project: Apache Cassandra
>          Issue Type: Bug
>          Components: Feature/Dynamic Data Masking, Transactional Cluster Metadata
>            Reporter: sepuri sai krishna
>            Assignee: sepuri sai krishna
>            Priority: Normal
>             Fix For: 6.x
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> h2. Summary
> {{ColumnMask.Serializer.deserialize}} has two independent defects that make any table with a
> dynamic data masked column unreadable from a serialized {{ClusterMetadata}}. A node that has taken
> a cluster metadata snapshot then fails to start.
> h2. Details
> In {{src/java/org/apache/cassandra/cql3/functions/masking/ColumnMask.java}}:
> {code:java}
> List<AbstractType<?>> argTypes = new ArrayList<>(numArgs + 1);
> argTypes.set(0, columnType);                                             // (1)
> ByteBuffer[] partialArgValues = new ByteBuffer[numArgs];
> for (int i = 0; i < numArgs; i++)
> {
>     AbstractType<?> argType = CQLTypeParser.parse(keyspace, in.readUTF(), types);
>     argTypes.set(i + 1,  argType);                                       // (1)
>     boolean valuePresent = in.readBoolean();
>     partialArgValues[i] = valuePresent ? null : ByteBufferUtil.readWithVIntLength(in);  // (2)
> }
> {code}
> # {{new ArrayList<>(numArgs + 1)}} sets the *capacity*, not the size, so the list is empty and {{set(0, ..)}}
> throws {{IndexOutOfBoundsException}}. This fires for every masked column, including
> {{MASKED WITH DEFAULT}}.
> # The ternary is inverted: the argument is discarded when present, and a read is attempted when
> absent. Besides losing the value this desynchronises the input stream, corrupting every field
> deserialized afterwards. With defect (1) fixed in isolation, the same tests fail with
> {{EOFException: EOF after 43 bytes out of 1024}}.
> h2. Impact
> {{ColumnMask.serializer}} is reached through
> {{ColumnMetadata -> TableMetadata -> Tables -> KeyspaceMetadata -> DistributedSchema -> ClusterMetadata}}.
> Ordinary DDL does not hit it, because schema changes propagate as transformations that each node
> replays locally. It is reached when the schema is embedded in a serialized {{ClusterMetadata}},
> i.e. for cluster metadata snapshots, which are read back on startup
> ({{LocalLog.replayPersisted}}) and when a lagging peer catches up from the CMS.
> Restarting a node that has a masked column and a stored snapshot fails during startup:
> {code}
> java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
> 	at java.base/java.util.ArrayList.set(ArrayList.java:470)
> 	at org.apache.cassandra.cql3.functions.masking.ColumnMask$Serializer.deserialize(ColumnMask.java:318)
> 	at org.apache.cassandra.schema.ColumnMetadata$Serializer.deserialize(ColumnMetadata.java:819)
> 	at org.apache.cassandra.schema.TableMetadata$Serializer.deserialize(TableMetadata.java:2184)
> 	at org.apache.cassandra.schema.Tables$Serializer.deserialize(Tables.java:317)
> 	at org.apache.cassandra.schema.KeyspaceMetadata$Serializer.deserialize(KeyspaceMetadata.java:526)
> 	at org.apache.cassandra.schema.DistributedSchema$Serializer.deserialize(DistributedSchema.java:487)
> 	at org.apache.cassandra.tcm.ClusterMetadata$Serializer.deserialize(ClusterMetadata.java:1326)
> 	at org.apache.cassandra.tcm.MetadataSnapshots.fromBytes(MetadataSnapshots.java:64)
> 	at org.apache.cassandra.tcm.MetadataSnapshots$SystemKeyspaceMetadataSnapshots.getLatestSnapshot(MetadataSnapshots.java:131)
> 	at org.apache.cassandra.tcm.log.SystemKeyspaceStorage.getPersistedLogState(SystemKeyspaceStorage.java:133)
> 	at org.apache.cassandra.tcm.log.LocalLog.replayPersisted(LocalLog.java:584)
> 	at org.apache.cassandra.tcm.Startup.initializeAsNonCmsNode(Startup.java:185)
> 	at org.apache.cassandra.tcm.Startup.initialize(Startup.java:120)
> {code}
> Note that {{MetadataSnapshots.getSnapshot}} only catches {{IOException}}, so this unchecked
> exception propagates rather than degrading to a null snapshot.
> h2. Reproduction
> {code}
> CREATE TABLE ks.t (k int PRIMARY KEY, v text MASKED WITH mask_inner(2, 1));
> nodetool cms snapshot
> # restart the node -> startup fails with the stack above
> {code}
> h2. Affected versions
> Present on {{cassandra-6.0}} and {{trunk}}. Not present on {{cassandra-5.0}}, which has no
> {{ColumnMask.Serializer}}; dynamic data masking shipped in 5.0 but the schema is only serialized
> into cluster metadata from 6.0 onwards. Patch is therefore based on {{cassandra-6.0}}.
> h2. Tests
> Existing DDM tests do not cover this because none of them force a metadata snapshot. Added:
> * three round-trip cases in {{SchemaMetadataSerializationTest}} covering a masking function with no
> partial arguments, with partial arguments, and with a null partial argument
> * a new dtest {{ColumnMaskMetadataSnapshotTest}} covering snapshot read-back and node restart
> All fail before the patch and pass after it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org