[jira] [Created] (CASSANDRA-21659) Adding a large amount of schema allocates excessively

["Patrick McFadin (Jira)" <ji...@apache.org>]

Patrick McFadin created CASSANDRA-21659:
-------------------------------------------

             Summary: Adding a large amount of schema allocates excessively
                 Key: CASSANDRA-21659
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-21659
             Project: Apache Cassandra
          Issue Type: Improvement
          Components: Cluster/Schema
            Reporter: Patrick McFadin


Every schema change allocates in proportion to the whole schema rather than to what changed.
h2. What

Keyspaces.diff and Tables.diff build created/dropped with filter(), which removes
every non-matching entry from a copy one at a time. On a CREATE TABLE no keyspace
is created or dropped, so Keyspaces.filter falls into withoutKsTablesViews and
removes every table in the cluster individually – twice per diff. Tables.diff
then calls TableMetadata.compare for every surviving table, and compare has no
identity fast path.

Several diffs run per DDL: AlterSchemaStatement:194 and :200, AlterSchema:160,
DistributedSchema:213, :260 and :325.
h2. Change
 * Collect created/dropped directly instead of filtering whole collections.
 * Skip comparing entries carried over by reference. Untouched TableMetadata are
reference-identical across a schema change (Tables.Builder.add stores the
instance verbatim), and x.compare(x) is empty by construction, so identity is
exact here, not an approximation.

No signature, format or API changes.
h2. Result

New KeyspacesDiffScalingTest measures allocation for adding one table at 400 and
3200 existing tables and asserts the growth ratio. Allocation rather than elapsed
time: counted exactly rather than sampled, independent of GC timing, and it is
what produces the failure (a GC wall, not gradual slowdown).
|| ||400 tables||3200 tables||growth||
|before|292,552 B|3,774,864 B|12.9x|
|after|3,944 B|3,944 B|1.0x|

Before is superlinear because the BTreeMap teardown is O(N log N).
h2. Scope – read before benchmarking

This removes the allocation term, not the scan. Time stays O(N) per diff and bulk
creation stays quadratic in wall-clock. What it removes is the allocation that
drives the GC wall. Wall-clock at small N will barely move.
h2. Tests

New KeyspacesDiffScalingTest: 2 scaling assertions, 3 correctness guards. Verified
red before and green after, and re-verified the final test fails with the fix
stashed. Regression: schema 27 suites/130 tests, tcm 13/55. checkstyle and
checkstyle-test clean.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org