You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "pmcfadin (via GitHub)" <gi...@apache.org> on 2026/09/07 03:16:22 UTC
[PR] CASSANDRA-21660: Back Tables with a persistent map so schema changes do not copy the collection [cassandra]
pmcfadin opened a new pull request, #5122:
URL: https://github.com/apache/cassandra/pull/5122
[CASSANDRA-21660](https://issues.apache.org/jira/browse/CASSANDRA-21660)
> **Stacked on [#5121](https://github.com/apache/cassandra/pull/5121) (CASSANDRA-21659).** That commit is included here because both touch `Tables.java`. Review only the second commit; the diff against #5121 is `Tables.java` plus one new test.
**What**
`Tables` keeps three `ImmutableMap`s and rebuilds all of them on every mutation: `with()` is `builder().add(this).add(table).build()`, `withSwapped()` is `without().with()`. A single `CREATE TABLE` therefore copies every existing table into three fresh maps more than once, and `Builder.add` recomputes index metadata — including a `TargetParser.parse` — for every indexed table while doing so.
CEP-21 converted the outer collection, `Keyspaces`, to a persistent `BTreeMap`: TCM holds schema at several epochs at once, so copy-on-write would copy the world per epoch. The collections nested inside a keyspace were not converted. This does `Tables`, which is the one deployments have thousands of.
**Change**
* Back the three maps with `BTreeMap`, so `with`/`without`/`withSwapped` update rather than copy.
* Maintain index tables incrementally, keyed off `IndexMetadata.name` — provably the same key, since `indexTableName` is `base + '.' + index.name` and `indexName` is the substring after the dot. Removal no longer builds a whole index `TableMetadata` just to read its name back off it.
* `Builder` keeps the duplicate-name rejection `ImmutableMap.Builder.build()` provided; a persistent map would otherwise overwrite silently.
**Result**
Allocation per operation:
| operation | 400 before | 3200 before | growth | 400 after | 3200 after | growth |
|---|---|---|---|---|---|---|
| `with` | 218,951 B | 1,687,617 B | 7.7x | 5,728 B | 5,920 B | 1.0x |
| `withSwapped` | 425,824 B | 3,349,656 B | 7.9x | 1,936 B | 2,448 B | 1.3x |
| `without` | 213,896 B | 1,682,368 B | 7.9x | 928 B | 1,200 B | 1.3x |
The residual 1.3x is the O(log n) spine a persistent map allocates per update — the mechanism, not a defect.
**⚠️ Requires a clean build**
`Tables.indexTables()` narrows from `ImmutableMap` to `Map`. That is binary incompatible, and ant's timestamp-based javac will not recompile callers whose source did not change. With a stale `build/` you get a `NoSuchMethodError` thrown inside `SecondaryIndexManager.createIndex`, which `LocalLog.processPendingInternal` catches and logs as "Could not process the entry" — leaving an index silently unregistered and surfacing later as an apparently unrelated NPE in a different test. Run `ant clean` first. No source change can prevent this.
**Tests**
New `TablesScalingTest`: 3 allocation assertions, 4 correctness guards (add/remove/replace semantics; iteration deterministic and agreeing with lookup by both name and id). Like #5121's test it asserts an allocation *ratio* rather than an absolute byte count, so it needs no re-tuning per JDK or machine.
Regression on a clean build: `org.apache.cassandra.schema` 29 suites, `org.apache.cassandra.tcm` 13 suites, `org.apache.cassandra.index` 3 suites / 61 tests — 0 failures. `ant checkstyle` and `ant checkstyle-test` clean.
**Worth a separate ticket**
`LocalLog.processPendingInternal` catches `Throwable`, logs "Could not process the entry" and continues, leaving a half-applied schema and no failed operation. Pre-existing TCM behaviour, but it turned a hard linkage error into a mystery here and would do the same to an operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-21660: Back Tables with a persistent map so schema changes do not copy the collection [cassandra]
Posted by "pmcfadin (via GitHub)" <gi...@apache.org>.
pmcfadin commented on PR #5122:
URL: https://github.com/apache/cassandra/pull/5122#issuecomment-5578709802
Superseded by a single combined patch for CASSANDRA-21664, which carries this commit unchanged as one of its first three commits. Closing in favour of that PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-21660: Back Tables with a persistent map so schema changes do not copy the collection [cassandra]
Posted by "pmcfadin (via GitHub)" <gi...@apache.org>.
pmcfadin commented on PR #5122:
URL: https://github.com/apache/cassandra/pull/5122#issuecomment-5564968935
JMH numbers, as requested. Benchmark committed here at `test/microbench/org/apache/cassandra/test/microbench/SchemaChangeBench.java`.
```
ant clean && ant build-test
ant microbench -Dbenchmark.name=SchemaChangeBench \
-Djmh.args="-prof gc -f 1 -wi 3 -i 5 -r 1 -w 1"
```
Baseline is `8cfc3a67be`, tip of `cassandra-6.0`. "21659 only" is the parent commit, so the last column isolates what *this* commit adds.
**`gc.alloc.rate.norm` — B/op**
| operation | N | baseline | 21659 only | + this patch |
|---|---|---|---|---|
| `Tables.with` | 400 | 203,464 | 205,264 | **1,064** |
| `Tables.with` | 3200 | 1,604,643 | 1,612,299 | **1,256** |
| `Tables.withSwapped` | 400 | 426,638 | 426,798 | **1,872** |
| `Tables.withSwapped` | 3200 | 3,358,689 | 3,355,044 | **2,384** |
| `Tables.without` | 400 | 212,536 | 213,066 | **872** |
| `Tables.without` | 3200 | 1,677,875 | 1,675,907 | **1,144** |
**Average time — ns/op**
| operation | N | baseline | 21659 only | + this patch |
|---|---|---|---|---|
| `Tables.with` | 3200 | 434,600 | 460,318 | **1,040** |
| `Tables.withSwapped` | 3200 | 879,889 | 958,776 | **1,299** |
| `Tables.without` | 3200 | 443,244 | 493,279 | **347** |
Allocation growth across an 8x larger collection goes from ~7.9x to 1.18x — the residual being the O(log n) spine a persistent map allocates per update. Time improves 400–1200x, since the operation stops copying the collection at all.
---
**A trade-off this surfaced, which I would rather raise than have a reviewer find.**
Backing `Tables` with a `BTreeMap` makes by-`TableId` lookup a binary search over `TableId.compareTo` (~108 ns) instead of a hash lookup (~8.6 ns). `TablesDiff.diff` performs 2N such lookups, so `Keyspaces.diff` gets slower in wall-clock:
| operation | N | baseline | 21659 only | + this patch |
|---|---|---|---|---|
| `Keyspaces.diff` | 3200 | 1,434,360 ns | 54,966 ns | 689,622 ns |
Allocation is unaffected (still flat at ~3,740 B/op). And this patch is still a clear net win against the base it merges into — 2x faster diff and ~400x faster `Tables` mutation — but it gives back most of CASSANDRA-21659's diff-time improvement.
The natural follow-up: both maps are now sorted by `TableId`, so `TablesDiff.diff` can be a linear merge-join over two sorted collections instead of N point lookups, removing the log factor entirely and beating both columns. I did not fold that into this patch because it is a change to diffing rather than to the collection, and belongs with CASSANDRA-21659's code. Happy to do it here instead if reviewers prefer the two to land together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] CASSANDRA-21660: Back Tables with a persistent map so schema changes do not copy the collection [cassandra]
Posted by "pmcfadin (via GitHub)" <gi...@apache.org>.
pmcfadin closed pull request #5122: CASSANDRA-21660: Back Tables with a persistent map so schema changes do not copy the collection
URL: https://github.com/apache/cassandra/pull/5122
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org