[PR] CASSANDRA-21615 UCS never compacts tiny high-density SSTables streamed by repair (6.0 branch) [cassandra]

["millerjp (via GitHub)" <gi...@apache.org>]

millerjp opened a new pull request, #5214:
URL: https://github.com/apache/cassandra/pull/5214

   The datastax/cassandra fix by @blambov (datastax/cassandra@5a6f917bc3) rebased onto cassandra-6.0 branch, keeping the original author. I only changed what would not compile. This branch has no CompactionRealm, so the table partition count is read from TableMetrics in ShardManager.create, and the ShardManagerReplicaAware changes were dropped because that class does not exist on this branch. On this branch the new ShardManagerNoDisks constructor argument also had to be passed in ParallelizedTasksTest and in testSplitSSTablesInRanges in ShardManagerTest. Those are compile fixes only, the tests are unchanged.
   
   Also added the same unit test and repair dtests as #5213 with minor changes to make it compile on 6.0. The dtests failed on this branch without the big fix, with the same tiny sstable growth per repair round, so 6.0 has the same problem on big.
   
   The big fix is the same as on 5.0 As suggested by @driftx on the ticket the count is read from getEstimatedPartitionSize().count() which works on both formats with a fallback to estimatedKeys() if for whatever reason partition sizes are null or 0 (unlikely, but thought it best to add this just in case).
   
   ShardManagerTest, UnifiedCompactionStrategyTest, ShardedCompactionWriterTest, ParallelizedTasksTest and ControllerTest all pass. UnifiedCompactionDensitiesTest passes on the default config and the two repair dtests pass on bti and big.
   
   Co-authored-by: @blambov
   
   A PR for trunk will follow.
   
   Fixes https://issues.apache.org/jira/browse/CASSANDRA-21615 on 6.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org

Re: [PR] CASSANDRA-21615 UCS never compacts tiny high-density SSTables streamed by repair (6.0 branch) [cassandra]

["millerjp (via GitHub)" <gi...@apache.org>]

millerjp commented on PR #5214:
URL: https://github.com/apache/cassandra/pull/5214#issuecomment-6445086648

   Brought up to the latest 5.0 changes in #5213, as one forward port commit on current `cassandra-6.0`. Where 6.0 differs from 5.0:
   
   - `splitSSTablesInShards` and `combineSetsWithCommonElement` already exist on 6.0, from the parallelized UCS work, so the port uses those. 6.0's `ShardManagerTrivial` overrides the split and returns one group, as the 5.0 version did.
   - `maybeUpdateShardManager` checks the `ClusterMetadata` epoch and the partition count estimate. It keeps 6.0's short circuit for local keyspaces, which still rebuild when the estimate changes.
   - The `enterDisk` fix also narrows a parallel task's operation range: the first task on a later disk covers (previous disk boundary, end] instead of (minimum, end]. No input is lost, because the split enters the disk holding the smallest first token.
   - `UnifiedCompactionHarryTest` is rewritten for the new Harry API, with the same schema, options and checks. The second visit to each partition deletes a row, deletes the partition, updates, or overwrites, so compaction reconciles tombstones as well.
   - `MaxSSTablesPerShardFactorSchemaUpgradeTest` is not ported. It covers 5.0.7 upgrade and rollback, which has no 6.0 equivalent.
   - NEWS goes in the 6.0 section, and the upgrade note says 6.0-alpha3 or later, or 5.0.10 or later. CHANGES goes first under "Merged from 5.0".
   - Small test changes: `getMaximumTokenForSplitting()` replaces `getMaximumToken()`, and three 6.0-only `ShardManagerNoDisks` callers pass the estimate.
   
   The unit tests for the touched classes pass on the default and latest configs. UnifiedCompactionDensitiesTest and UnifiedCompactionHarryTest pass with big and bti.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org